2024-03-28T00:52:18.976 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-28T00:52:18.976 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252 --owner mchangir --description fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} --name mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --verbose -- /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/orig.config.yaml 2024-03-28T00:52:19.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T00:52:19.110 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252 branch: wip-mchangir-testing-main-20240318.032620 description: fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} email: mchangir@redhat.com first_in_suite: false job_id: '7625252' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 3645 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 132/256 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdDFkbJ8neEy/kRcin/a5EEuG94zZpcsnyExwll9pO+AuwMq9VHSxbQV0SjsbKA7FlR9ml23QSJHNtvur+RG9Q= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAWl1fIfCuM29ayfT64fpl1N5+l5xGMs9311mNHxmwUSXAtKLdyj78swsnTh3QWOPEO/KcZnRNnde6WJJnQsBg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - workunit: clients: client.0: - client/test.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_03:45:23 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-28T00:52:19.110 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-28T00:52:19.111 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-28T00:52:19.111 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-28T00:52:19.112 INFO:teuthology.task.internal:Checking packages... 2024-03-28T00:52:19.138 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-28T00:52:19.138 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-28T00:52:19.139 INFO:teuthology.packaging:ref: None 2024-03-28T00:52:19.139 INFO:teuthology.packaging:tag: None 2024-03-28T00:52:19.139 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-28T00:52:19.139 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:52:19.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-28T00:52:19.357 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-28T00:52:19.358 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-28T00:52:19.365 INFO:teuthology.task.internal:no buildpackages task found 2024-03-28T00:52:19.365 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-28T00:52:19.373 INFO:teuthology.task.internal:Saving configuration 2024-03-28T00:52:19.389 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-28T00:52:19.395 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T00:52:19.418 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 00:47:25.364107', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdDFkbJ8neEy/kRcin/a5EEuG94zZpcsnyExwll9pO+AuwMq9VHSxbQV0SjsbKA7FlR9ml23QSJHNtvur+RG9Q='} 2024-03-28T00:52:19.443 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 00:47:25.365789', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAWl1fIfCuM29ayfT64fpl1N5+l5xGMs9311mNHxmwUSXAtKLdyj78swsnTh3QWOPEO/KcZnRNnde6WJJnQsBg='} 2024-03-28T00:52:19.443 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-28T00:52:19.449 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-28T00:52:19.449 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-28T00:52:19.449 INFO:teuthology.run_tasks:Running task console_log... 2024-03-28T00:52:19.520 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d26135e50>, signals=[15]) 2024-03-28T00:52:19.521 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-28T00:52:19.527 INFO:teuthology.task.internal:Opening connections... 2024-03-28T00:52:19.527 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T00:52:19.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:52:19.636 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T00:52:19.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:52:19.724 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-28T00:52:19.730 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-03-28T00:52:19.748 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-03-28T00:52:19.748 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-03-28T00:52:19.805 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-03-28T00:52:19.805 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-03-28T00:52:19.805 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-03-28T00:52:19.805 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-03-28T00:52:19.805 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-03-28T00:52:19.805 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-03-28T00:52:19.805 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T00:52:19.806 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-03-28T00:52:19.806 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-03-28T00:52:19.806 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T00:52:19.806 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-03-28T00:52:19.806 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T00:52:19.806 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T00:52:19.806 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T00:52:19.807 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-03-28T00:52:19.825 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-03-28T00:52:19.842 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-03-28T00:52:19.842 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Stream" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="9" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="9" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el9" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:LOGO="fedora-logo-icon" 2024-03-28T00:52:19.899 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T00:52:19.900 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://centos.org/" 2024-03-28T00:52:19.900 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T00:52:19.900 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T00:52:19.900 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T00:52:19.900 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-03-28T00:52:19.924 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-28T00:52:19.934 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-28T00:52:19.939 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-28T00:52:19.939 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-03-28T00:52:19.942 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-03-28T00:52:19.957 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-28T00:52:19.963 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-28T00:52:19.963 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-03-28T00:52:19.999 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-03-28T00:52:20.024 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-28T00:52:20.082 INFO:teuthology.run_tasks:Running task kernel... 2024-03-28T00:52:20.097 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-28T00:52:20.097 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-28T00:52:20.098 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T00:52:20.098 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T00:52:20.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:52:20.114 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:52:20.114 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T00:52:20.114 DEBUG:teuthology.orchestra.run.smithi194:> uname -r 2024-03-28T00:52:20.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:52:20.115 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:52:20.115 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T00:52:20.116 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-03-28T00:52:20.170 INFO:teuthology.orchestra.run.smithi194.stdout:5.14.0-361.el9.x86_64 2024-03-28T00:52:20.171 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:52:20.171 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T00:52:20.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:52:20.175 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-361.el9.x86_64 2024-03-28T00:52:20.175 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:52:20.175 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T00:52:20.175 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:52:20.313 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum upgrade -y linux-firmware 2024-03-28T00:52:20.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum upgrade -y linux-firmware 2024-03-28T00:52:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-28T00:52:22.486 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-28T00:52:23.900 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-03-28T00:52:23.945 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-28T00:52:24.707 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-28T00:52:25.327 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.0 MB 00:01 2024-03-28T00:52:27.283 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-28T00:52:27.820 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-28T00:52:31.742 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.1 MB 00:00 2024-03-28T00:52:32.012 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-28T00:52:33.268 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-03-28T00:52:33.396 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 16 kB 00:00 2024-03-28T00:52:34.663 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-28T00:52:35.017 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-28T00:52:39.610 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-28T00:52:40.172 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-28T00:52:41.927 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-03-28T00:52:41.939 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-03-28T00:52:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-03-28T00:52:41.939 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 1 Package 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 375 M 2024-03-28T00:52:41.940 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-03-28T00:52:42.530 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-03-28T00:52:42.543 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-03-28T00:52:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-03-28T00:52:42.543 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-03-28T00:52:42.543 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-03-28T00:52:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T00:52:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T00:52:42.544 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-03-28T00:52:42.544 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-03-28T00:52:42.544 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 1 Package 2024-03-28T00:52:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T00:52:42.544 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 375 M 2024-03-28T00:52:42.545 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-03-28T00:52:52.656 INFO:teuthology.orchestra.run.smithi033.stdout:linux-firmware-20240219-143.el9.noarch.rpm 37 MB/s | 375 MB 00:10 2024-03-28T00:52:52.657 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-03-28T00:52:52.657 INFO:teuthology.orchestra.run.smithi033.stdout:Total 37 MB/s | 375 MB 00:10 2024-03-28T00:52:52.658 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-03-28T00:52:52.764 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-03-28T00:52:52.764 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-03-28T00:52:54.410 INFO:teuthology.orchestra.run.smithi194.stdout:linux-firmware-20240219-143.el9.noarch.rpm 30 MB/s | 375 MB 00:12 2024-03-28T00:52:54.414 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-03-28T00:52:54.415 INFO:teuthology.orchestra.run.smithi194.stdout:Total 30 MB/s | 375 MB 00:12 2024-03-28T00:52:54.415 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-03-28T00:52:54.502 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-03-28T00:52:54.502 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-03-28T00:52:55.273 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-03-28T00:52:55.273 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-03-28T00:52:56.875 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-03-28T00:52:56.875 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-03-28T00:53:01.892 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-03-28T00:53:02.170 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T00:53:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:53:03.230 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-03-28T00:53:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:53:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T00:53:03.503 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T00:53:03.584 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:53:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:53:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T00:53:03.848 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-03-28T00:53:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T00:53:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T00:53:03.848 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-03-28T00:53:04.135 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-28T00:53:04.135 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:53:04.462 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T00:53:04.463 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T00:53:04.465 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi033.front.sepia.ceph.com to complete... 2024-03-28T00:53:04.876 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:53:04.876 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T00:53:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:53:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T00:53:05.289 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-03-28T00:53:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T00:53:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T00:53:05.289 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-03-28T00:53:05.558 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-28T00:53:05.558 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:53:05.923 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T00:53:05.923 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T00:53:05.925 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi194.front.sepia.ceph.com to complete... 2024-03-28T00:53:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28 00:53:25 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T00:53:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:FINISHED --2024-03-28 00:53:25-- 2024-03-28T00:53:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:Total wall clock time: 21s 2024-03-28T00:53:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:Downloaded: 1 files, 321M in 21s (15.5 MB/s) 2024-03-28T00:53:25.529 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-28T00:53:25.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T00:53:27.562 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying... ######################################## 2024-03-28T00:53:27.619 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing... ######################################## 2024-03-28T00:53:27.704 INFO:teuthology.orchestra.run.smithi033.stdout:Updating / installing... 2024-03-28T00:53:35.544 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T00:53:36.482 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28 00:53:36 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T00:53:36.482 INFO:teuthology.orchestra.run.smithi194.stderr:FINISHED --2024-03-28 00:53:36-- 2024-03-28T00:53:36.482 INFO:teuthology.orchestra.run.smithi194.stderr:Total wall clock time: 31s 2024-03-28T00:53:36.482 INFO:teuthology.orchestra.run.smithi194.stderr:Downloaded: 1 files, 321M in 30s (10.6 MB/s) 2024-03-28T00:53:36.485 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-28T00:53:36.485 DEBUG:teuthology.orchestra.run.smithi194:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T00:53:38.414 INFO:teuthology.orchestra.run.smithi194.stdout:Verifying... ######################################## 2024-03-28T00:53:38.537 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing... ######################################## 2024-03-28T00:53:38.617 INFO:teuthology.orchestra.run.smithi194.stdout:Updating / installing... 2024-03-28T00:53:46.086 INFO:teuthology.orchestra.run.smithi194.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T00:53:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T00:53:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T00:54:14.939 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T00:54:14.939 DEBUG:teuthology.orchestra.run.smithi033:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T00:54:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T00:54:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T00:54:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T00:54:15.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T00:54:15.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T00:54:15.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T00:54:15.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T00:54:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T00:54:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T00:54:15.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T00:54:15.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T00:54:15.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T00:54:15.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T00:54:15.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T00:54:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T00:54:15.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T00:54:15.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T00:54:15.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T00:54:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T00:54:15.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T00:54:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T00:54:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T00:54:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T00:54:15.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T00:54:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T00:54:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T00:54:15.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T00:54:15.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T00:54:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T00:54:15.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T00:54:15.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T00:54:15.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T00:54:15.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T00:54:15.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T00:54:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T00:54:15.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T00:54:15.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T00:54:15.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T00:54:15.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T00:54:15.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T00:54:15.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T00:54:15.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T00:54:15.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T00:54:15.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T00:54:15.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T00:54:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T00:54:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T00:54:15.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T00:54:15.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T00:54:15.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T00:54:15.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T00:54:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T00:54:15.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T00:54:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T00:54:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T00:54:15.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T00:54:15.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T00:54:15.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T00:54:15.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T00:54:15.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T00:54:15.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T00:54:15.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T00:54:15.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T00:54:15.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T00:54:15.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T00:54:15.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T00:54:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T00:54:15.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T00:54:15.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T00:54:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T00:54:15.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T00:54:15.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T00:54:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T00:54:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T00:54:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T00:54:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T00:54:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T00:54:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T00:54:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T00:54:15.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T00:54:15.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T00:54:15.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T00:54:15.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T00:54:15.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T00:54:15.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T00:54:15.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T00:54:15.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T00:54:15.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T00:54:15.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T00:54:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T00:54:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T00:54:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T00:54:15.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T00:54:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T00:54:15.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T00:54:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T00:54:15.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T00:54:15.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T00:54:15.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T00:54:15.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T00:54:15.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T00:54:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T00:54:15.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T00:54:15.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T00:54:15.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T00:54:15.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T00:54:15.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T00:54:15.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T00:54:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T00:54:15.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T00:54:15.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T00:54:15.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T00:54:15.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T00:54:15.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T00:54:15.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T00:54:15.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T00:54:15.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T00:54:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T00:54:15.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T00:54:15.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T00:54:15.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T00:54:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T00:54:15.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T00:54:15.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T00:54:15.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T00:54:15.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T00:54:15.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T00:54:15.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T00:54:15.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T00:54:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T00:54:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T00:54:15.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T00:54:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T00:54:15.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T00:54:15.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T00:54:15.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T00:54:15.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T00:54:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T00:54:15.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T00:54:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T00:54:15.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T00:54:15.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T00:54:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T00:54:15.162 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T00:54:15.163 DEBUG:teuthology.orchestra.run.smithi033:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T00:54:15.186 INFO:teuthology.orchestra.run.smithi033.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T00:54:15.186 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T00:54:15.186 INFO:teuthology.orchestra.run.smithi033.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:for file in vmlinuz System.map config; do 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout: fi 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:done 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2024-03-28T00:54:15.187 INFO:teuthology.orchestra.run.smithi033.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T00:54:15.188 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T00:54:15.188 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T00:54:15.188 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2024-03-28T00:54:15.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.06 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 68.el9 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Group : Unspecified 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 8264254 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T00:54:15.270 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : builder@centos.org 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T00:54:15.271 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T00:54:15.273 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T00:54:15.273 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 6.8.0-g1df0c7e8a235 2024-03-28T00:54:15.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T00:54:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-03-28T00:54:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T00:54:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-03-28T00:54:17.367 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-03-28T00:54:17.382 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.5zzKdbvUCE 2024-03-28T00:54:17.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5zzKdbvUCE 2024-03-28T00:54:17.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.5zzKdbvUCE 2024-03-28T00:54:17.594 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.5zzKdbvUCE is 6KB 2024-03-28T00:54:17.643 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.5zzKdbvUCE 2024-03-28T00:54:17.658 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T00:54:17.726 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T00:54:17.726 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T00:54:17.726 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T00:54:17.726 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T00:54:17.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T00:54:17.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-03-28T00:54:22.326 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi194.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T00:54:22.326 DEBUG:teuthology.orchestra.run.smithi194:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T00:54:22.390 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T00:54:22.391 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T00:54:22.392 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T00:54:22.393 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T00:54:22.394 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T00:54:22.395 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T00:54:22.396 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T00:54:22.397 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T00:54:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T00:54:22.399 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T00:54:22.400 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T00:54:22.401 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T00:54:22.402 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T00:54:22.403 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T00:54:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T00:54:22.405 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T00:54:22.406 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T00:54:22.407 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T00:54:22.408 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T00:54:22.409 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T00:54:22.410 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T00:54:22.411 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T00:54:22.412 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T00:54:22.413 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T00:54:22.414 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T00:54:22.415 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T00:54:22.416 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T00:54:22.417 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T00:54:22.418 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T00:54:22.419 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T00:54:22.420 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T00:54:22.421 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T00:54:22.422 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T00:54:22.423 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T00:54:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T00:54:22.425 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T00:54:22.426 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T00:54:22.427 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T00:54:22.428 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T00:54:22.429 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T00:54:22.430 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T00:54:22.431 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T00:54:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T00:54:22.433 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T00:54:22.434 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T00:54:22.435 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T00:54:22.436 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T00:54:22.437 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T00:54:22.438 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T00:54:22.439 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T00:54:22.440 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T00:54:22.441 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T00:54:22.442 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T00:54:22.443 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T00:54:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T00:54:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T00:54:22.446 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T00:54:22.447 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T00:54:22.448 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T00:54:22.449 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T00:54:22.450 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T00:54:22.451 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T00:54:22.452 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T00:54:22.453 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T00:54:22.454 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T00:54:22.455 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T00:54:22.456 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T00:54:22.457 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T00:54:22.458 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T00:54:22.459 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T00:54:22.460 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T00:54:22.461 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T00:54:22.462 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T00:54:22.463 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T00:54:22.464 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T00:54:22.465 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T00:54:22.466 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T00:54:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T00:54:22.468 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T00:54:22.469 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T00:54:22.470 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T00:54:22.471 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T00:54:22.472 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T00:54:22.473 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T00:54:22.474 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T00:54:22.475 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T00:54:22.476 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T00:54:22.477 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T00:54:22.478 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T00:54:22.479 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T00:54:22.480 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T00:54:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T00:54:22.482 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T00:54:22.483 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T00:54:22.484 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T00:54:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T00:54:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T00:54:22.487 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T00:54:22.488 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T00:54:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T00:54:22.490 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T00:54:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T00:54:22.492 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T00:54:22.493 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T00:54:22.494 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T00:54:22.495 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T00:54:22.496 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T00:54:22.497 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T00:54:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T00:54:22.499 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T00:54:22.500 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T00:54:22.501 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T00:54:22.502 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T00:54:22.503 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T00:54:22.504 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T00:54:22.505 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T00:54:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T00:54:22.507 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T00:54:22.508 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T00:54:22.509 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T00:54:22.510 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T00:54:22.511 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T00:54:22.512 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T00:54:22.513 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T00:54:22.514 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T00:54:22.515 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T00:54:22.516 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T00:54:22.517 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T00:54:22.518 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T00:54:22.519 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T00:54:22.520 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T00:54:22.521 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T00:54:22.522 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T00:54:22.523 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T00:54:22.524 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T00:54:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T00:54:22.526 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T00:54:22.527 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T00:54:22.528 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T00:54:22.529 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T00:54:22.530 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T00:54:22.531 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T00:54:22.532 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T00:54:22.533 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T00:54:22.534 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T00:54:22.535 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T00:54:22.536 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T00:54:22.537 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T00:54:22.538 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T00:54:22.539 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T00:54:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T00:54:22.541 INFO:teuthology.orchestra.run.smithi194.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T00:54:22.545 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T00:54:22.545 DEBUG:teuthology.orchestra.run.smithi194:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout:fi 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout:for file in vmlinuz System.map config; do 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T00:54:22.567 INFO:teuthology.orchestra.run.smithi194.stdout: fi 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:done 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:fi 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T00:54:22.568 INFO:teuthology.orchestra.run.smithi194.stdout:fi 2024-03-28T00:54:22.569 DEBUG:teuthology.orchestra.run.smithi194:> sudo rpm -qi grub2-tools 2024-03-28T00:54:22.643 INFO:teuthology.orchestra.run.smithi194.stdout:Name : grub2-tools 2024-03-28T00:54:22.643 INFO:teuthology.orchestra.run.smithi194.stdout:Epoch : 1 2024-03-28T00:54:22.643 INFO:teuthology.orchestra.run.smithi194.stdout:Version : 2.06 2024-03-28T00:54:22.643 INFO:teuthology.orchestra.run.smithi194.stdout:Release : 68.el9 2024-03-28T00:54:22.643 INFO:teuthology.orchestra.run.smithi194.stdout:Architecture: x86_64 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Group : Unspecified 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Size : 8264254 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:License : GPLv3+ 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Packager : builder@centos.org 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Vendor : CentOS 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Summary : Support tools for GRUB. 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:Description : 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T00:54:22.644 INFO:teuthology.orchestra.run.smithi194.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T00:54:22.645 INFO:teuthology.orchestra.run.smithi194.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T00:54:22.645 INFO:teuthology.orchestra.run.smithi194.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T00:54:22.645 INFO:teuthology.orchestra.run.smithi194.stdout:hardware devices. 2024-03-28T00:54:22.645 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T00:54:22.645 INFO:teuthology.orchestra.run.smithi194.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T00:54:22.646 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T00:54:22.646 INFO:teuthology.task.kernel:Updating grub on smithi194 to boot 6.8.0-g1df0c7e8a235 2024-03-28T00:54:22.646 DEBUG:teuthology.orchestra.run.smithi194:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T00:54:23.245 INFO:teuthology.orchestra.run.smithi194.stderr:Generating grub configuration file ... 2024-03-28T00:54:24.599 INFO:teuthology.orchestra.run.smithi194.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T00:54:24.647 INFO:teuthology.orchestra.run.smithi194.stderr:done 2024-03-28T00:54:24.650 DEBUG:teuthology.orchestra.run.smithi194:> mktemp 2024-03-28T00:54:24.665 INFO:teuthology.orchestra.run.smithi194.stdout:/tmp/tmp.HmoCH3vTSy 2024-03-28T00:54:24.665 DEBUG:teuthology.orchestra.run.smithi194:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HmoCH3vTSy 2024-03-28T00:54:24.743 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0666 /tmp/tmp.HmoCH3vTSy 2024-03-28T00:54:24.883 DEBUG:teuthology.orchestra.remote:smithi194:/tmp/tmp.HmoCH3vTSy is 6KB 2024-03-28T00:54:24.931 DEBUG:teuthology.orchestra.run.smithi194:> rm -fr /tmp/tmp.HmoCH3vTSy 2024-03-28T00:54:24.945 DEBUG:teuthology.orchestra.run.smithi194:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T00:54:25.009 INFO:teuthology.orchestra.run.smithi194.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T00:54:25.009 INFO:teuthology.orchestra.run.smithi194.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T00:54:25.009 INFO:teuthology.orchestra.run.smithi194.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T00:54:25.010 INFO:teuthology.orchestra.run.smithi194.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T00:54:25.011 DEBUG:teuthology.orchestra.run.smithi194:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T00:54:25.163 DEBUG:teuthology.orchestra.run.smithi194:> sudo shutdown -r now 2024-03-28T00:54:47.902 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-28T00:54:47.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-03-28T00:54:47.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:54:55.173 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-28T00:54:55.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-28T00:54:55.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:55:06.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-03-28T00:55:13.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-03-28T00:55:15.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-03-28T00:55:15.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:55:22.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-28T00:55:22.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:55:25.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-03-28T00:55:33.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-03-28T00:55:37.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-28T00:55:37.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:55:45.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-03-28T00:55:45.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:55:48.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-03-28T00:56:03.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-03-28T00:56:03.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:56:04.282 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-03-28T00:56:04.870 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-03-28T00:56:04.870 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-28T00:56:04.870 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-03-28T00:56:04.911 INFO:teuthology.orchestra.run.smithi033.stdout:6.8.0-g1df0c7e8a235 2024-03-28T00:56:04.911 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T00:56:04.911 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T00:56:04.912 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T00:56:05.913 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-28T00:56:05.913 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T00:56:06.099 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-03-28T00:56:06.143 DEBUG:teuthology.parallel:result is None 2024-03-28T00:56:09.694 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-03-28T00:56:10.406 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-28T00:56:10.406 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-28T00:56:10.406 DEBUG:teuthology.orchestra.run.smithi194:> uname -r 2024-03-28T00:56:10.442 INFO:teuthology.orchestra.run.smithi194.stdout:6.8.0-g1df0c7e8a235 2024-03-28T00:56:10.443 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T00:56:10.443 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T00:56:10.443 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T00:56:11.444 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-28T00:56:11.444 DEBUG:teuthology.orchestra.run.smithi194:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T00:56:11.563 INFO:teuthology.orchestra.run.smithi194.stdout:ttyS1 2024-03-28T00:56:11.599 DEBUG:teuthology.parallel:result is None 2024-03-28T00:56:11.599 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-28T00:56:11.611 INFO:teuthology.task.internal:Creating test directory... 2024-03-28T00:56:11.611 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T00:56:11.617 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T00:56:11.655 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-28T00:56:11.662 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-28T00:56:11.672 INFO:teuthology.task.internal:Creating archive directory... 2024-03-28T00:56:11.672 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T00:56:11.697 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T00:56:11.753 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-28T00:56:11.760 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-28T00:56:11.761 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T00:56:11.768 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T00:56:11.852 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:56:11.853 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:56:11.890 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:56:11.894 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:56:11.899 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-28T00:56:11.906 INFO:teuthology.task.internal:Configuring sudo... 2024-03-28T00:56:11.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T00:56:11.911 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T00:56:11.988 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-28T00:56:11.998 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-28T00:56:11.998 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T00:56:12.003 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T00:56:12.045 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:56:12.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:56:12.253 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:56:12.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:56:12.407 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T00:56:12.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T00:56:12.486 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:56:12.561 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:56:12.640 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:56:12.704 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:56:12.800 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T00:56:12.800 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T00:56:12.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-03-28T00:56:12.884 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-03-28T00:56:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T00:56:13.026 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T00:56:13.274 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-28T00:56:13.283 INFO:teuthology.task.internal:Starting timer... 2024-03-28T00:56:13.283 INFO:teuthology.run_tasks:Running task pcp... 2024-03-28T00:56:13.294 INFO:teuthology.run_tasks:Running task selinux... 2024-03-28T00:56:13.304 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-28T00:56:13.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-03-28T00:56:13.546 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-03-28T00:56:13.552 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2024-03-28T00:56:13.664 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2024-03-28T00:56:13.668 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-28T00:56:13.668 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-03-28T00:56:13.787 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-03-28T00:56:13.788 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2024-03-28T00:56:13.840 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2024-03-28T00:56:13.840 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive'} 2024-03-28T00:56:13.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T00:56:13.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:56:13.921 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T00:56:13.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:56:13.996 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-28T00:56:13.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-03-28T00:56:14.091 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2024-03-28T00:56:14.186 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-28T00:56:14.545 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-28T00:56:14.550 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-28T00:56:14.674 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-28T00:56:14.690 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-28T00:56:14.691 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-28T01:02:50.971 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2024-03-28T01:02:50.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-03-28T01:02:50.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T01:02:51.123 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-03-28T01:02:51.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-03-28T01:02:51.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-28T01:02:51.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T01:02:51.423 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-03-28T01:02:51.575 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-28T01:02:51.575 INFO:teuthology.run_tasks:Running task clock... 2024-03-28T01:02:51.586 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-28T01:02:51.586 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T01:02:51.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T01:02:51.594 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T01:02:51.594 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T01:02:51.696 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T01:02:51.698 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T01:02:51.751 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T01:02:51.752 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T01:02:51.847 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2024-03-28T01:02:51.854 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-03-28T01:02:51.888 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2024-03-28T01:02:51.901 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-03-28T01:02:51.940 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T01:02:51.957 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T01:02:51.991 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T01:02:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T01:02:52.125 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-03-28T01:02:52.135 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T01:02:52.135 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-03-28T01:02:52.135 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.135 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.135 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.135 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.150 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-03-28T01:02:52.159 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T01:02:52.159 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-03-28T01:02:52.159 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.159 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.160 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.160 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:02:52.161 INFO:teuthology.run_tasks:Running task install... 2024-03-28T01:02:52.170 DEBUG:teuthology.task.install:project ceph 2024-03-28T01:02:52.170 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-28T01:02:52.170 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-28T01:02:52.170 INFO:teuthology.task.install:Using flavor: default 2024-03-28T01:02:52.175 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-28T01:02:52.175 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-28T01:02:52.176 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T01:02:52.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T01:02:52.178 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T01:02:52.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T01:02:52.321 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T01:02:52.322 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T01:02:52.397 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T01:02:52.397 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T01:02:52.491 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T01:02:52.492 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:02:52.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T01:02:52.503 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T01:02:52.504 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:02:52.504 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T01:02:52.598 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T01:02:52.598 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T01:02:52.604 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T01:02:52.604 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T01:02:52.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T01:02:52.743 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T01:02:52.876 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T01:02:52.877 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T01:02:52.965 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2024-03-28T01:02:52.966 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-03-28T01:02:52.969 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2024-03-28T01:02:52.971 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-03-28T01:02:53.322 INFO:teuthology.orchestra.run.smithi194.stdout:74 files removed 2024-03-28T01:02:53.347 INFO:teuthology.orchestra.run.smithi033.stdout:74 files removed 2024-03-28T01:02:53.360 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T01:02:53.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T01:02:54.358 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 170 kB/s | 83 kB 00:00 2024-03-28T01:02:54.383 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 171 kB/s | 83 kB 00:00 2024-03-28T01:02:54.817 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 28 kB/s | 10 kB 00:00 2024-03-28T01:02:54.909 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 23 kB/s | 10 kB 00:00 2024-03-28T01:02:55.301 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-28T01:02:55.409 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-03-28T01:02:55.988 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-28T01:02:56.063 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-28T01:02:57.998 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - BaseOS 2.8 MB/s | 2.0 MB 00:00 2024-03-28T01:02:58.131 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - BaseOS 2.6 MB/s | 2.0 MB 00:00 2024-03-28T01:02:59.260 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-03-28T01:02:59.284 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-28T01:03:01.519 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-28T01:03:05.987 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.1 MB 00:00 2024-03-28T01:03:08.257 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 16 kB 00:00 2024-03-28T01:03:10.786 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 21 MB 00:02 2024-03-28T01:03:16.506 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-28T01:03:19.165 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T01:03:19.167 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T01:03:19.189 INFO:teuthology.orchestra.run.smithi194.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T01:03:19.191 INFO:teuthology.orchestra.run.smithi194.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T01:03:19.192 INFO:teuthology.orchestra.run.smithi194.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T01:03:19.194 INFO:teuthology.orchestra.run.smithi194.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T01:03:19.196 INFO:teuthology.orchestra.run.smithi194.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T01:03:19.200 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T01:03:19.202 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T01:03:19.290 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-03-28T01:03:19.312 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-28T01:03:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-03-28T01:03:19.312 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-28T01:03:19.312 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-03-28T01:03:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T01:03:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T01:03:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T01:03:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-03-28T01:03:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T01:03:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T01:03:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T01:03:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T01:03:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T01:03:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T01:03:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T01:03:19.322 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T01:03:19.323 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T01:03:19.324 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T01:03:19.325 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T01:03:19.326 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:03:19.327 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-03-28T01:03:19.327 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-28T01:03:19.327 INFO:teuthology.orchestra.run.smithi194.stdout:Install 150 Packages 2024-03-28T01:03:19.327 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 24 Packages 2024-03-28T01:03:19.327 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:03:19.327 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 225 M 2024-03-28T01:03:19.327 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-03-28T01:03:20.457 INFO:teuthology.orchestra.run.smithi194.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-03-28T01:03:20.766 INFO:teuthology.orchestra.run.smithi194.stdout:(2/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 8.7 MB/s | 5.4 MB 00:00 2024-03-28T01:03:20.817 INFO:teuthology.orchestra.run.smithi194.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 2.9 MB/s | 145 kB 00:00 2024-03-28T01:03:21.025 INFO:teuthology.orchestra.run.smithi194.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 2.3 MB 00:00 2024-03-28T01:03:21.066 INFO:teuthology.orchestra.run.smithi194.stdout:(5/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.4 MB/s | 890 kB 00:00 2024-03-28T01:03:21.142 INFO:teuthology.orchestra.run.smithi194.stdout:(6/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-28T01:03:22.117 INFO:teuthology.orchestra.run.smithi194.stdout:(7/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 17 MB/s | 17 MB 00:00 2024-03-28T01:03:22.659 INFO:teuthology.orchestra.run.smithi194.stdout:(8/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-03-28T01:03:22.826 INFO:teuthology.orchestra.run.smithi194.stdout:(9/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 14 MB/s | 9.9 MB 00:00 2024-03-28T01:03:22.852 INFO:teuthology.orchestra.run.smithi194.stdout:(10/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el 132 kB/s | 25 kB 00:00 2024-03-28T01:03:22.991 INFO:teuthology.orchestra.run.smithi194.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 1.6 MB/s | 220 kB 00:00 2024-03-28T01:03:23.092 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 856 kB/s | 19 MB 00:22 2024-03-28T01:03:23.101 INFO:teuthology.orchestra.run.smithi194.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 248 kB/s | 27 kB 00:00 2024-03-28T01:03:23.210 INFO:teuthology.orchestra.run.smithi194.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 291 kB/s | 31 kB 00:00 2024-03-28T01:03:23.427 INFO:teuthology.orchestra.run.smithi194.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 3.4 MB/s | 746 kB 00:00 2024-03-28T01:03:23.536 INFO:teuthology.orchestra.run.smithi194.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.5 MB/s | 163 kB 00:00 2024-03-28T01:03:23.652 INFO:teuthology.orchestra.run.smithi194.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.1 MB/s | 127 kB 00:00 2024-03-28T01:03:23.861 INFO:teuthology.orchestra.run.smithi194.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 2.4 MB/s | 501 kB 00:00 2024-03-28T01:03:24.770 INFO:teuthology.orchestra.run.smithi194.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 5.7 MB/s | 5.1 MB 00:00 2024-03-28T01:03:24.870 INFO:teuthology.orchestra.run.smithi194.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 456 kB/s | 45 kB 00:00 2024-03-28T01:03:25.154 INFO:teuthology.orchestra.run.smithi194.stdout:(20/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 4.1 MB/s | 20 MB 00:05 2024-03-28T01:03:25.187 INFO:teuthology.orchestra.run.smithi194.stdout:(21/174): python3-ceph-common-19.0.0-2145.g3be9 415 kB/s | 131 kB 00:00 2024-03-28T01:03:25.254 INFO:teuthology.orchestra.run.smithi194.stdout:(22/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.6 MB/s | 162 kB 00:00 2024-03-28T01:03:25.279 INFO:teuthology.orchestra.run.smithi194.stdout:(23/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.4 MB/s | 321 kB 00:00 2024-03-28T01:03:25.346 INFO:teuthology.orchestra.run.smithi194.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 99 kB 00:00 2024-03-28T01:03:25.380 INFO:teuthology.orchestra.run.smithi194.stdout:(25/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 2.9 MB/s | 298 kB 00:00 2024-03-28T01:03:25.947 INFO:teuthology.orchestra.run.smithi194.stdout:(26/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 15 MB/s | 48 MB 00:03 2024-03-28T01:03:25.964 INFO:teuthology.orchestra.run.smithi194.stdout:(27/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 138 kB/s | 85 kB 00:00 2024-03-28T01:03:26.006 INFO:teuthology.orchestra.run.smithi194.stdout:(28/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-28T01:03:26.040 INFO:teuthology.orchestra.run.smithi194.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 455 kB/s | 34 kB 00:00 2024-03-28T01:03:26.065 INFO:teuthology.orchestra.run.smithi194.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.3 MB/s | 139 kB 00:00 2024-03-28T01:03:26.140 INFO:teuthology.orchestra.run.smithi194.stdout:(31/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 4.1 MB/s | 3.1 MB 00:00 2024-03-28T01:03:26.223 INFO:teuthology.orchestra.run.smithi194.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 3.0 MB/s | 248 kB 00:00 2024-03-28T01:03:26.332 INFO:teuthology.orchestra.run.smithi194.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 457 kB/s | 49 kB 00:00 2024-03-28T01:03:26.441 INFO:teuthology.orchestra.run.smithi194.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 153 kB/s | 16 kB 00:00 2024-03-28T01:03:26.708 INFO:teuthology.orchestra.run.smithi194.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-03-28T01:03:26.766 INFO:teuthology.orchestra.run.smithi194.stdout:(36/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 2.5 MB/s | 1.8 MB 00:00 2024-03-28T01:03:26.817 INFO:teuthology.orchestra.run.smithi194.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 6.8 MB/s | 748 kB 00:00 2024-03-28T01:03:26.842 INFO:teuthology.orchestra.run.smithi194.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 246 kB/s | 18 kB 00:00 2024-03-28T01:03:27.000 INFO:teuthology.orchestra.run.smithi194.stdout:(39/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 472 kB/s | 264 kB 00:00 2024-03-28T01:03:27.134 INFO:teuthology.orchestra.run.smithi194.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 920 kB 00:00 2024-03-28T01:03:27.176 INFO:teuthology.orchestra.run.smithi194.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-03-28T01:03:27.210 INFO:teuthology.orchestra.run.smithi194.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 513 kB/s | 106 kB 00:00 2024-03-28T01:03:27.243 INFO:teuthology.orchestra.run.smithi194.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 307 kB 00:00 2024-03-28T01:03:27.268 INFO:teuthology.orchestra.run.smithi194.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 32 kB 00:00 2024-03-28T01:03:27.469 INFO:teuthology.orchestra.run.smithi194.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-28T01:03:27.494 INFO:teuthology.orchestra.run.smithi194.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 79 kB/s | 17 kB 00:00 2024-03-28T01:03:27.519 INFO:teuthology.orchestra.run.smithi194.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 41 kB/s | 10 kB 00:00 2024-03-28T01:03:27.553 INFO:teuthology.orchestra.run.smithi194.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-28T01:03:27.586 INFO:teuthology.orchestra.run.smithi194.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 684 kB/s | 44 kB 00:00 2024-03-28T01:03:27.637 INFO:teuthology.orchestra.run.smithi194.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 661 kB 00:00 2024-03-28T01:03:27.670 INFO:teuthology.orchestra.run.smithi194.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.1 MB/s | 578 kB 00:00 2024-03-28T01:03:27.687 INFO:teuthology.orchestra.run.smithi194.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 500 kB/s | 46 kB 00:00 2024-03-28T01:03:27.721 INFO:teuthology.orchestra.run.smithi194.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 282 kB 00:00 2024-03-28T01:03:27.754 INFO:teuthology.orchestra.run.smithi194.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-28T01:03:27.788 INFO:teuthology.orchestra.run.smithi194.stdout:(55/174): lua-5.4.4-4.el9.x86_64.rpm 1.9 MB/s | 187 kB 00:00 2024-03-28T01:03:27.813 INFO:teuthology.orchestra.run.smithi194.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 381 kB/s | 34 kB 00:00 2024-03-28T01:03:28.030 INFO:teuthology.orchestra.run.smithi194.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-03-28T01:03:28.047 INFO:teuthology.orchestra.run.smithi194.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 358 kB/s | 78 kB 00:00 2024-03-28T01:03:28.114 INFO:teuthology.orchestra.run.smithi194.stdout:(59/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-03-28T01:03:28.167 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.1 MB 00:01 2024-03-28T01:03:28.247 INFO:teuthology.orchestra.run.smithi194.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-28T01:03:28.273 INFO:teuthology.orchestra.run.smithi194.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 126 kB/s | 24 kB 00:00 2024-03-28T01:03:28.298 INFO:teuthology.orchestra.run.smithi194.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 109 kB/s | 18 kB 00:00 2024-03-28T01:03:28.340 INFO:teuthology.orchestra.run.smithi194.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 788 kB/s | 32 kB 00:00 2024-03-28T01:03:28.398 INFO:teuthology.orchestra.run.smithi194.stdout:(64/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 144 kB 00:00 2024-03-28T01:03:28.574 INFO:teuthology.orchestra.run.smithi194.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-28T01:03:28.599 INFO:teuthology.orchestra.run.smithi194.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 919 kB/s | 217 kB 00:00 2024-03-28T01:03:28.616 INFO:teuthology.orchestra.run.smithi194.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 858 kB/s | 174 kB 00:00 2024-03-28T01:03:28.658 INFO:teuthology.orchestra.run.smithi194.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-03-28T01:03:28.683 INFO:teuthology.orchestra.run.smithi194.stdout:(69/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-28T01:03:28.708 INFO:teuthology.orchestra.run.smithi194.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-03-28T01:03:28.734 INFO:teuthology.orchestra.run.smithi194.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-03-28T01:03:28.759 INFO:teuthology.orchestra.run.smithi194.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 990 kB/s | 47 kB 00:00 2024-03-28T01:03:28.793 INFO:teuthology.orchestra.run.smithi194.stdout:(73/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-28T01:03:28.817 INFO:teuthology.orchestra.run.smithi194.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 519 kB/s | 43 kB 00:00 2024-03-28T01:03:29.160 INFO:teuthology.orchestra.run.smithi194.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-03-28T01:03:29.184 INFO:teuthology.orchestra.run.smithi194.stdout:(76/174): python3-toml-0.10.2-6.el9.noarch.rpm 112 kB/s | 39 kB 00:00 2024-03-28T01:03:29.210 INFO:teuthology.orchestra.run.smithi194.stdout:(77/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 111 kB/s | 41 kB 00:00 2024-03-28T01:03:29.260 INFO:teuthology.orchestra.run.smithi194.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2024-03-28T01:03:29.285 INFO:teuthology.orchestra.run.smithi194.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 56 kB 00:00 2024-03-28T01:03:29.310 INFO:teuthology.orchestra.run.smithi194.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 638 kB/s | 63 kB 00:00 2024-03-28T01:03:29.361 INFO:teuthology.orchestra.run.smithi194.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-28T01:03:29.386 INFO:teuthology.orchestra.run.smithi194.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-28T01:03:29.411 INFO:teuthology.orchestra.run.smithi194.stdout:(83/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 443 kB/s | 44 kB 00:00 2024-03-28T01:03:29.445 INFO:teuthology.orchestra.run.smithi194.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-28T01:03:29.495 INFO:teuthology.orchestra.run.smithi194.stdout:(85/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-28T01:03:29.520 INFO:teuthology.orchestra.run.smithi194.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-28T01:03:29.554 INFO:teuthology.orchestra.run.smithi194.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 187 kB 00:00 2024-03-28T01:03:29.713 INFO:teuthology.orchestra.run.smithi194.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 990 kB/s | 187 kB 00:00 2024-03-28T01:03:29.738 INFO:teuthology.orchestra.run.smithi194.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-03-28T01:03:29.771 INFO:teuthology.orchestra.run.smithi194.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-28T01:03:29.814 INFO:teuthology.orchestra.run.smithi194.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-03-28T01:03:29.851 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-03-28T01:03:29.855 INFO:teuthology.orchestra.run.smithi194.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 185 kB/s | 21 kB 00:00 2024-03-28T01:03:29.881 INFO:teuthology.orchestra.run.smithi194.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-03-28T01:03:29.886 INFO:teuthology.orchestra.run.smithi194.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:29.910 INFO:teuthology.orchestra.run.smithi194.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:30.069 INFO:teuthology.orchestra.run.smithi194.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:30.100 INFO:teuthology.orchestra.run.smithi194.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:30.439 INFO:teuthology.orchestra.run.smithi194.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 1.6 MB/s | 911 kB 00:00 2024-03-28T01:03:30.498 INFO:teuthology.orchestra.run.smithi194.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 2.5 MB/s | 1.6 MB 00:00 2024-03-28T01:03:30.499 INFO:teuthology.orchestra.run.smithi194.stdout:[MIRROR] postgresql-private-libs-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:30.631 INFO:teuthology.orchestra.run.smithi194.stdout:(96/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.1 MB/s | 4.1 MB 00:00 2024-03-28T01:03:30.633 INFO:teuthology.orchestra.run.smithi194.stdout:[MIRROR] postgresql-server-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:30.673 INFO:teuthology.orchestra.run.smithi194.stdout:(97/174): postgresql-private-libs-13.14-1.el9.x 572 kB/s | 133 kB 00:00 2024-03-28T01:03:30.732 INFO:teuthology.orchestra.run.smithi194.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-03-28T01:03:30.791 INFO:teuthology.orchestra.run.smithi194.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-03-28T01:03:30.849 INFO:teuthology.orchestra.run.smithi194.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-03-28T01:03:30.874 INFO:teuthology.orchestra.run.smithi194.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 435 kB/s | 36 kB 00:00 2024-03-28T01:03:30.933 INFO:teuthology.orchestra.run.smithi194.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 574 kB/s | 47 kB 00:00 2024-03-28T01:03:30.958 INFO:teuthology.orchestra.run.smithi194.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-28T01:03:31.075 INFO:teuthology.orchestra.run.smithi194.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 10 MB/s | 5.8 MB 00:00 2024-03-28T01:03:31.226 INFO:teuthology.orchestra.run.smithi194.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-03-28T01:03:31.536 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-28T01:03:31.610 INFO:teuthology.orchestra.run.smithi194.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-03-28T01:03:31.718 INFO:teuthology.orchestra.run.smithi194.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 855 kB/s | 548 kB 00:00 2024-03-28T01:03:31.805 INFO:teuthology.orchestra.run.smithi194.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 534 kB/s | 309 kB 00:00 2024-03-28T01:03:31.868 INFO:teuthology.orchestra.run.smithi194.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 76 kB/s | 19 kB 00:00 2024-03-28T01:03:31.910 INFO:teuthology.orchestra.run.smithi194.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 249 kB/s | 26 kB 00:00 2024-03-28T01:03:32.027 INFO:teuthology.orchestra.run.smithi194.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 582 kB/s | 67 kB 00:00 2024-03-28T01:03:32.161 INFO:teuthology.orchestra.run.smithi194.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-28T01:03:32.186 INFO:teuthology.orchestra.run.smithi194.stdout:(113/174): liboath-2.6.7-2.el9.x86_64.rpm 153 kB/s | 49 kB 00:00 2024-03-28T01:03:32.220 INFO:teuthology.orchestra.run.smithi194.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 790 kB/s | 151 kB 00:00 2024-03-28T01:03:32.329 INFO:teuthology.orchestra.run.smithi194.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-03-28T01:03:32.354 INFO:teuthology.orchestra.run.smithi194.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2024-03-28T01:03:32.429 INFO:teuthology.orchestra.run.smithi194.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 324 kB/s | 32 kB 00:00 2024-03-28T01:03:32.454 INFO:teuthology.orchestra.run.smithi194.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-03-28T01:03:32.547 INFO:teuthology.orchestra.run.smithi194.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-03-28T01:03:32.580 INFO:teuthology.orchestra.run.smithi194.stdout:(120/174): python3-asyncssh-2.13.2-5.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-03-28T01:03:32.622 INFO:teuthology.orchestra.run.smithi194.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-03-28T01:03:32.656 INFO:teuthology.orchestra.run.smithi194.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-28T01:03:32.682 INFO:teuthology.orchestra.run.smithi194.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-03-28T01:03:32.748 INFO:teuthology.orchestra.run.smithi194.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-28T01:03:32.807 INFO:teuthology.orchestra.run.smithi194.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-03-28T01:03:32.883 INFO:teuthology.orchestra.run.smithi194.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-03-28T01:03:32.908 INFO:teuthology.orchestra.run.smithi194.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 68 kB/s | 11 kB 00:00 2024-03-28T01:03:32.933 INFO:teuthology.orchestra.run.smithi194.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 142 kB/s | 18 kB 00:00 2024-03-28T01:03:32.983 INFO:teuthology.orchestra.run.smithi194.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-03-28T01:03:33.009 INFO:teuthology.orchestra.run.smithi194.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-03-28T01:03:33.034 INFO:teuthology.orchestra.run.smithi194.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-03-28T01:03:33.126 INFO:teuthology.orchestra.run.smithi194.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-03-28T01:03:33.151 INFO:teuthology.orchestra.run.smithi194.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2024-03-28T01:03:33.168 INFO:teuthology.orchestra.run.smithi194.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 592 kB/s | 79 kB 00:00 2024-03-28T01:03:33.226 INFO:teuthology.orchestra.run.smithi194.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 580 kB/s | 58 kB 00:00 2024-03-28T01:03:33.285 INFO:teuthology.orchestra.run.smithi194.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-03-28T01:03:33.319 INFO:teuthology.orchestra.run.smithi194.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-03-28T01:03:33.336 INFO:teuthology.orchestra.run.smithi194.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 833 kB/s | 90 kB 00:00 2024-03-28T01:03:33.386 INFO:teuthology.orchestra.run.smithi194.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-03-28T01:03:33.419 INFO:teuthology.orchestra.run.smithi194.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-03-28T01:03:33.453 INFO:teuthology.orchestra.run.smithi194.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-03-28T01:03:33.487 INFO:teuthology.orchestra.run.smithi194.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 592 kB/s | 59 kB 00:00 2024-03-28T01:03:33.520 INFO:teuthology.orchestra.run.smithi194.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-03-28T01:03:33.604 INFO:teuthology.orchestra.run.smithi194.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 340 kB/s | 51 kB 00:00 2024-03-28T01:03:33.671 INFO:teuthology.orchestra.run.smithi194.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 598 kB/s | 90 kB 00:00 2024-03-28T01:03:33.704 INFO:teuthology.orchestra.run.smithi194.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-03-28T01:03:33.763 INFO:teuthology.orchestra.run.smithi194.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-03-28T01:03:33.780 INFO:teuthology.orchestra.run.smithi194.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-03-28T01:03:33.822 INFO:teuthology.orchestra.run.smithi194.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-28T01:03:33.947 INFO:teuthology.orchestra.run.smithi194.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-03-28T01:03:34.156 INFO:teuthology.orchestra.run.smithi194.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.1 MB/s | 321 kB 00:00 2024-03-28T01:03:34.440 INFO:teuthology.orchestra.run.smithi194.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 8.2 MB/s | 2.3 MB 00:00 2024-03-28T01:03:34.557 INFO:teuthology.orchestra.run.smithi194.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-03-28T01:03:34.615 INFO:teuthology.orchestra.run.smithi194.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 731 kB/s | 42 kB 00:00 2024-03-28T01:03:34.699 INFO:teuthology.orchestra.run.smithi194.stdout:(155/174): librados2-19.0.0-2145.g3be9a5c9.el9. 3.7 MB/s | 3.4 MB 00:00 2024-03-28T01:03:34.757 INFO:teuthology.orchestra.run.smithi194.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.7 MB/s | 248 kB 00:00 2024-03-28T01:03:34.808 INFO:teuthology.orchestra.run.smithi194.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 748 kB/s | 36 kB 00:00 2024-03-28T01:03:34.841 INFO:teuthology.orchestra.run.smithi194.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 505 kB/s | 71 kB 00:00 2024-03-28T01:03:34.866 INFO:teuthology.orchestra.run.smithi194.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2024-03-28T01:03:34.917 INFO:teuthology.orchestra.run.smithi194.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 469 kB/s | 23 kB 00:00 2024-03-28T01:03:34.950 INFO:teuthology.orchestra.run.smithi194.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-28T01:03:34.976 INFO:teuthology.orchestra.run.smithi194.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-28T01:03:35.009 INFO:teuthology.orchestra.run.smithi194.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-28T01:03:35.043 INFO:teuthology.orchestra.run.smithi194.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-28T01:03:35.068 INFO:teuthology.orchestra.run.smithi194.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-28T01:03:35.093 INFO:teuthology.orchestra.run.smithi194.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 582 kB/s | 29 kB 00:00 2024-03-28T01:03:35.160 INFO:teuthology.orchestra.run.smithi194.stdout:(167/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-03-28T01:03:35.269 INFO:teuthology.orchestra.run.smithi194.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2024-03-28T01:03:35.311 INFO:teuthology.orchestra.run.smithi194.stdout:(169/174): openssl-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 1.2 MB 00:00 2024-03-28T01:03:35.361 INFO:teuthology.orchestra.run.smithi194.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 633 kB/s | 31 kB 00:00 2024-03-28T01:03:35.478 INFO:teuthology.orchestra.run.smithi194.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.3 MB/s | 1.1 MB 00:00 2024-03-28T01:03:35.520 INFO:teuthology.orchestra.run.smithi194.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 1.8 MB/s | 465 kB 00:00 2024-03-28T01:03:35.587 INFO:teuthology.orchestra.run.smithi194.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.4 MB/s | 2.3 MB 00:00 2024-03-28T01:03:35.612 INFO:teuthology.orchestra.run.smithi194.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 561 kB/s | 74 kB 00:00 2024-03-28T01:03:35.621 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-03-28T01:03:35.621 INFO:teuthology.orchestra.run.smithi194.stdout:Total 14 MB/s | 225 MB 00:16 2024-03-28T01:03:37.271 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-03-28T01:03:37.569 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-28T01:03:37.649 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-03-28T01:03:37.649 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-03-28T01:03:40.066 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-03-28T01:03:40.066 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-03-28T01:03:40.372 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T01:03:40.373 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T01:03:40.396 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T01:03:40.398 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T01:03:40.399 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T01:03:40.401 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T01:03:40.404 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T01:03:40.407 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T01:03:40.409 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T01:03:40.504 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-03-28T01:03:40.530 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-03-28T01:03:40.530 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-03-28T01:03:40.530 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-03-28T01:03:40.530 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-03-28T01:03:40.530 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T01:03:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T01:03:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T01:03:40.533 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T01:03:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T01:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T01:03:40.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T01:03:40.537 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T01:03:40.538 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T01:03:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T01:03:40.540 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T01:03:40.541 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T01:03:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T01:03:40.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T01:03:40.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T01:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:03:40.546 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-03-28T01:03:40.546 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-03-28T01:03:40.546 INFO:teuthology.orchestra.run.smithi033.stdout:Install 150 Packages 2024-03-28T01:03:40.546 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 24 Packages 2024-03-28T01:03:40.546 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:03:40.546 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 225 M 2024-03-28T01:03:40.546 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-03-28T01:03:41.221 INFO:teuthology.orchestra.run.smithi033.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-28T01:03:41.971 INFO:teuthology.orchestra.run.smithi033.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.2 MB/s | 890 kB 00:00 2024-03-28T01:03:42.279 INFO:teuthology.orchestra.run.smithi033.stdout:(3/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 15 MB/s | 20 MB 00:01 2024-03-28T01:03:42.330 INFO:teuthology.orchestra.run.smithi033.stdout:(4/174): ceph-immutable-object-cache-19.0.0-214 406 kB/s | 145 kB 00:00 2024-03-28T01:03:42.480 INFO:teuthology.orchestra.run.smithi033.stdout:(5/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 12 MB/s | 2.3 MB 00:00 2024-03-28T01:03:42.815 INFO:teuthology.orchestra.run.smithi033.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 14 MB/s | 4.7 MB 00:00 2024-03-28T01:03:43.014 INFO:teuthology.orchestra.run.smithi033.stdout:(7/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-03-28T01:03:43.172 INFO:teuthology.orchestra.run.smithi033.stdout:(8/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-03-28T01:03:43.273 INFO:teuthology.orchestra.run.smithi033.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 254 kB/s | 25 kB 00:00 2024-03-28T01:03:43.823 INFO:teuthology.orchestra.run.smithi033.stdout:(10/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 17 MB/s | 17 MB 00:01 2024-03-28T01:03:43.890 INFO:teuthology.orchestra.run.smithi033.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 3.3 MB/s | 220 kB 00:00 2024-03-28T01:03:43.957 INFO:teuthology.orchestra.run.smithi033.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 412 kB/s | 27 kB 00:00 2024-03-28T01:03:44.024 INFO:teuthology.orchestra.run.smithi033.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 476 kB/s | 31 kB 00:00 2024-03-28T01:03:44.132 INFO:teuthology.orchestra.run.smithi033.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 6.8 MB/s | 746 kB 00:00 2024-03-28T01:03:44.190 INFO:teuthology.orchestra.run.smithi033.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 2.8 MB/s | 163 kB 00:00 2024-03-28T01:03:44.241 INFO:teuthology.orchestra.run.smithi033.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 2.5 MB/s | 127 kB 00:00 2024-03-28T01:03:44.400 INFO:teuthology.orchestra.run.smithi033.stdout:(17/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 7.1 MB/s | 9.9 MB 00:01 2024-03-28T01:03:44.459 INFO:teuthology.orchestra.run.smithi033.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 2.3 MB/s | 501 kB 00:00 2024-03-28T01:03:44.526 INFO:teuthology.orchestra.run.smithi033.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 688 kB/s | 45 kB 00:00 2024-03-28T01:03:44.592 INFO:teuthology.orchestra.run.smithi033.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 2.0 MB/s | 131 kB 00:00 2024-03-28T01:03:44.659 INFO:teuthology.orchestra.run.smithi033.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.4 MB/s | 162 kB 00:00 2024-03-28T01:03:44.785 INFO:teuthology.orchestra.run.smithi033.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 2.5 MB/s | 321 kB 00:00 2024-03-28T01:03:44.910 INFO:teuthology.orchestra.run.smithi033.stdout:(23/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 10 MB/s | 5.1 MB 00:00 2024-03-28T01:03:44.944 INFO:teuthology.orchestra.run.smithi033.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.8 MB/s | 298 kB 00:00 2024-03-28T01:03:44.986 INFO:teuthology.orchestra.run.smithi033.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.3 MB/s | 99 kB 00:00 2024-03-28T01:03:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-28T01:03:45.086 INFO:teuthology.orchestra.run.smithi033.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.2 MB/s | 170 kB 00:00 2024-03-28T01:03:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 516 kB/s | 34 kB 00:00 2024-03-28T01:03:45.275 INFO:teuthology.orchestra.run.smithi033.stdout:(29/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 11 MB/s | 3.1 MB 00:00 2024-03-28T01:03:45.312 INFO:teuthology.orchestra.run.smithi033.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 881 kB/s | 139 kB 00:00 2024-03-28T01:03:45.513 INFO:teuthology.orchestra.run.smithi033.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 7.6 MB/s | 1.8 MB 00:00 2024-03-28T01:03:45.621 INFO:teuthology.orchestra.run.smithi033.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.3 MB/s | 248 kB 00:00 2024-03-28T01:03:45.721 INFO:teuthology.orchestra.run.smithi033.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 498 kB/s | 49 kB 00:00 2024-03-28T01:03:45.813 INFO:teuthology.orchestra.run.smithi033.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 181 kB/s | 16 kB 00:00 2024-03-28T01:03:45.947 INFO:teuthology.orchestra.run.smithi033.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-03-28T01:03:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 1.6 MB/s | 264 kB 00:00 2024-03-28T01:03:46.039 INFO:teuthology.orchestra.run.smithi033.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-28T01:03:46.064 INFO:teuthology.orchestra.run.smithi033.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 220 kB/s | 18 kB 00:00 2024-03-28T01:03:46.323 INFO:teuthology.orchestra.run.smithi033.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 920 kB 00:00 2024-03-28T01:03:46.365 INFO:teuthology.orchestra.run.smithi033.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2024-03-28T01:03:46.390 INFO:teuthology.orchestra.run.smithi033.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-28T01:03:46.457 INFO:teuthology.orchestra.run.smithi033.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-28T01:03:46.482 INFO:teuthology.orchestra.run.smithi033.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 347 kB/s | 32 kB 00:00 2024-03-28T01:03:46.549 INFO:teuthology.orchestra.run.smithi033.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 268 kB/s | 17 kB 00:00 2024-03-28T01:03:46.617 INFO:teuthology.orchestra.run.smithi033.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-03-28T01:03:46.642 INFO:teuthology.orchestra.run.smithi033.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 115 kB/s | 10 kB 00:00 2024-03-28T01:03:46.667 INFO:teuthology.orchestra.run.smithi033.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-03-28T01:03:46.692 INFO:teuthology.orchestra.run.smithi033.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 921 kB/s | 44 kB 00:00 2024-03-28T01:03:46.751 INFO:teuthology.orchestra.run.smithi033.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2024-03-28T01:03:46.793 INFO:teuthology.orchestra.run.smithi033.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2024-03-28T01:03:46.818 INFO:teuthology.orchestra.run.smithi033.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 760 kB/s | 46 kB 00:00 2024-03-28T01:03:46.852 INFO:teuthology.orchestra.run.smithi033.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-28T01:03:46.886 INFO:teuthology.orchestra.run.smithi033.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-28T01:03:46.919 INFO:teuthology.orchestra.run.smithi033.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-28T01:03:46.936 INFO:teuthology.orchestra.run.smithi033.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 719 kB/s | 34 kB 00:00 2024-03-28T01:03:47.079 INFO:teuthology.orchestra.run.smithi033.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-28T01:03:47.120 INFO:teuthology.orchestra.run.smithi033.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-03-28T01:03:47.178 INFO:teuthology.orchestra.run.smithi033.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-28T01:03:47.313 INFO:teuthology.orchestra.run.smithi033.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-03-28T01:03:47.329 INFO:teuthology.orchestra.run.smithi033.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 182 kB/s | 24 kB 00:00 2024-03-28T01:03:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-03-28T01:03:47.379 INFO:teuthology.orchestra.run.smithi033.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 371 kB/s | 18 kB 00:00 2024-03-28T01:03:47.413 INFO:teuthology.orchestra.run.smithi033.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-28T01:03:47.438 INFO:teuthology.orchestra.run.smithi033.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 558 kB/s | 32 kB 00:00 2024-03-28T01:03:47.507 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T01:03:47.552 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T01:03:47.572 INFO:teuthology.orchestra.run.smithi033.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-28T01:03:47.588 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T01:03:47.597 INFO:teuthology.orchestra.run.smithi033.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-28T01:03:47.631 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T01:03:47.639 INFO:teuthology.orchestra.run.smithi033.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2024-03-28T01:03:47.664 INFO:teuthology.orchestra.run.smithi033.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-28T01:03:47.718 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T01:03:47.734 INFO:teuthology.orchestra.run.smithi033.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 599 kB/s | 41 kB 00:00 2024-03-28T01:03:47.773 INFO:teuthology.orchestra.run.smithi033.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 238 kB 00:00 2024-03-28T01:03:47.801 INFO:teuthology.orchestra.run.smithi033.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-28T01:03:47.822 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T01:03:47.832 INFO:teuthology.orchestra.run.smithi033.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-28T01:03:47.857 INFO:teuthology.orchestra.run.smithi033.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 846 kB/s | 47 kB 00:00 2024-03-28T01:03:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T01:03:47.883 INFO:teuthology.orchestra.run.smithi033.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 875 kB/s | 43 kB 00:00 2024-03-28T01:03:47.894 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T01:03:48.025 INFO:teuthology.orchestra.run.smithi033.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 310 kB/s | 39 kB 00:00 2024-03-28T01:03:48.060 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T01:03:48.309 INFO:teuthology.orchestra.run.smithi033.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-28T01:03:48.334 INFO:teuthology.orchestra.run.smithi033.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 137 kB/s | 41 kB 00:00 2024-03-28T01:03:48.376 INFO:teuthology.orchestra.run.smithi033.stdout:(77/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-03-28T01:03:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 300 kB 00:00 2024-03-28T01:03:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-28T01:03:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-03-28T01:03:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-28T01:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-03-28T01:03:48.586 INFO:teuthology.orchestra.run.smithi033.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-28T01:03:48.611 INFO:teuthology.orchestra.run.smithi033.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-28T01:03:48.614 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T01:03:48.645 INFO:teuthology.orchestra.run.smithi033.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-28T01:03:48.652 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T01:03:48.678 INFO:teuthology.orchestra.run.smithi033.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-03-28T01:03:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T01:03:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T01:03:48.887 INFO:teuthology.orchestra.run.smithi033.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 782 kB/s | 187 kB 00:00 2024-03-28T01:03:48.938 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T01:03:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T01:03:49.047 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T01:03:49.349 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T01:03:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T01:03:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T01:03:49.463 INFO:teuthology.orchestra.run.smithi033.stdout:(88/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 7.7 MB/s | 48 MB 00:06 2024-03-28T01:03:49.496 INFO:teuthology.orchestra.run.smithi033.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 321 kB/s | 195 kB 00:00 2024-03-28T01:03:49.538 INFO:teuthology.orchestra.run.smithi033.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 935 kB/s | 803 kB 00:00 2024-03-28T01:03:49.597 INFO:teuthology.orchestra.run.smithi033.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 216 kB/s | 21 kB 00:00 2024-03-28T01:03:49.600 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T01:03:49.636 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T01:03:49.655 INFO:teuthology.orchestra.run.smithi033.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 219 kB/s | 25 kB 00:00 2024-03-28T01:03:49.680 INFO:teuthology.orchestra.run.smithi033.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 577 kB/s | 125 kB 00:00 2024-03-28T01:03:49.685 INFO:teuthology.orchestra.run.smithi033.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:49.734 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T01:03:49.788 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T01:03:49.788 INFO:teuthology.orchestra.run.smithi033.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:49.812 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T01:03:49.874 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T01:03:49.887 INFO:teuthology.orchestra.run.smithi033.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:49.949 INFO:teuthology.orchestra.run.smithi033.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-03-28T01:03:50.128 INFO:teuthology.orchestra.run.smithi033.stdout:[MIRROR] postgresql-private-libs-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T01:03:50.164 INFO:teuthology.orchestra.run.smithi033.stdout:(95/174): postgresql-contrib-13.14-1.el9.x86_64 1.8 MB/s | 911 kB 00:00 2024-03-28T01:03:50.196 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T01:03:50.198 INFO:teuthology.orchestra.run.smithi033.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-03-28T01:03:50.237 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T01:03:50.248 INFO:teuthology.orchestra.run.smithi033.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.9 MB/s | 245 kB 00:00 2024-03-28T01:03:50.278 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T01:03:50.340 INFO:teuthology.orchestra.run.smithi033.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 4.1 MB 00:00 2024-03-28T01:03:50.374 INFO:teuthology.orchestra.run.smithi033.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-28T01:03:50.433 INFO:teuthology.orchestra.run.smithi033.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 631 kB/s | 36 kB 00:00 2024-03-28T01:03:50.466 INFO:teuthology.orchestra.run.smithi033.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-03-28T01:03:50.500 INFO:teuthology.orchestra.run.smithi033.stdout:(102/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-03-28T01:03:50.534 INFO:teuthology.orchestra.run.smithi033.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-28T01:03:50.559 INFO:teuthology.orchestra.run.smithi033.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 386 kB/s | 22 kB 00:00 2024-03-28T01:03:50.677 INFO:teuthology.orchestra.run.smithi033.stdout:(105/174): postgresql-server-13.14-1.el9.x86_64 11 MB/s | 5.8 MB 00:00 2024-03-28T01:03:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T01:03:50.837 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T01:03:50.876 INFO:teuthology.orchestra.run.smithi033.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-28T01:03:50.925 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T01:03:50.993 INFO:teuthology.orchestra.run.smithi033.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 980 kB/s | 309 kB 00:00 2024-03-28T01:03:51.035 INFO:teuthology.orchestra.run.smithi033.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-28T01:03:51.069 INFO:teuthology.orchestra.run.smithi033.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 101 kB/s | 19 kB 00:00 2024-03-28T01:03:51.111 INFO:teuthology.orchestra.run.smithi033.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 348 kB/s | 26 kB 00:00 2024-03-28T01:03:51.150 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T01:03:51.164 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T01:03:51.178 INFO:teuthology.orchestra.run.smithi033.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-28T01:03:51.194 INFO:teuthology.orchestra.run.smithi033.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 387 kB/s | 49 kB 00:00 2024-03-28T01:03:51.292 INFO:teuthology.orchestra.run.smithi033.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-28T01:03:51.481 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T01:03:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-03-28T01:03:51.529 INFO:teuthology.orchestra.run.smithi033.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-03-28T01:03:51.571 INFO:teuthology.orchestra.run.smithi033.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-03-28T01:03:51.588 INFO:teuthology.orchestra.run.smithi033.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 477 kB/s | 43 kB 00:00 2024-03-28T01:03:51.605 INFO:teuthology.orchestra.run.smithi033.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 430 kB/s | 32 kB 00:00 2024-03-28T01:03:51.630 INFO:teuthology.orchestra.run.smithi033.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-03-28T01:03:51.639 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T01:03:51.688 INFO:teuthology.orchestra.run.smithi033.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-28T01:03:51.722 INFO:teuthology.orchestra.run.smithi033.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-28T01:03:51.756 INFO:teuthology.orchestra.run.smithi033.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-28T01:03:51.760 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T01:03:51.781 INFO:teuthology.orchestra.run.smithi033.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-03-28T01:03:51.803 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T01:03:51.815 INFO:teuthology.orchestra.run.smithi033.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-28T01:03:51.839 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T01:03:51.896 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T01:03:51.907 INFO:teuthology.orchestra.run.smithi033.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-03-28T01:03:51.932 INFO:teuthology.orchestra.run.smithi033.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-28T01:03:52.005 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T01:03:52.016 INFO:teuthology.orchestra.run.smithi033.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-03-28T01:03:52.041 INFO:teuthology.orchestra.run.smithi033.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-03-28T01:03:52.066 INFO:teuthology.orchestra.run.smithi033.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-03-28T01:03:52.076 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T01:03:52.100 INFO:teuthology.orchestra.run.smithi033.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 234 kB/s | 19 kB 00:00 2024-03-28T01:03:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T01:03:52.142 INFO:teuthology.orchestra.run.smithi033.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2024-03-28T01:03:52.200 INFO:teuthology.orchestra.run.smithi033.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-03-28T01:03:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T01:03:52.234 INFO:teuthology.orchestra.run.smithi033.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-03-28T01:03:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T01:03:52.260 INFO:teuthology.orchestra.run.smithi033.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 676 kB/s | 79 kB 00:00 2024-03-28T01:03:52.289 INFO:teuthology.orchestra.run.smithi033.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 669 kB/s | 58 kB 00:00 2024-03-28T01:03:52.309 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T01:03:52.338 INFO:teuthology.orchestra.run.smithi033.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-28T01:03:52.363 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T01:03:52.395 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T01:03:52.397 INFO:teuthology.orchestra.run.smithi033.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 121 kB/s | 16 kB 00:00 2024-03-28T01:03:52.414 INFO:teuthology.orchestra.run.smithi033.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 378 kB/s | 28 kB 00:00 2024-03-28T01:03:52.440 INFO:teuthology.orchestra.run.smithi033.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 597 kB/s | 90 kB 00:00 2024-03-28T01:03:52.473 INFO:teuthology.orchestra.run.smithi033.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-03-28T01:03:52.487 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T01:03:52.506 INFO:teuthology.orchestra.run.smithi033.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-28T01:03:52.531 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T01:03:52.532 INFO:teuthology.orchestra.run.smithi033.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 646 kB/s | 59 kB 00:00 2024-03-28T01:03:52.557 INFO:teuthology.orchestra.run.smithi033.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-03-28T01:03:52.582 INFO:teuthology.orchestra.run.smithi033.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 682 kB/s | 51 kB 00:00 2024-03-28T01:03:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T01:03:52.622 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T01:03:52.633 INFO:teuthology.orchestra.run.smithi033.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-03-28T01:03:52.658 INFO:teuthology.orchestra.run.smithi033.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 900 kB/s | 90 kB 00:00 2024-03-28T01:03:52.700 INFO:teuthology.orchestra.run.smithi033.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-28T01:03:52.725 INFO:teuthology.orchestra.run.smithi033.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-03-28T01:03:52.735 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T01:03:52.750 INFO:teuthology.orchestra.run.smithi033.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-28T01:03:52.792 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T01:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T01:03:52.917 INFO:teuthology.orchestra.run.smithi033.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-03-28T01:03:53.060 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T01:03:53.076 INFO:teuthology.orchestra.run.smithi033.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-03-28T01:03:53.120 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T01:03:53.285 INFO:teuthology.orchestra.run.smithi033.stdout:(152/174): librados2-19.0.0-2145.g3be9a5c9.el9. 6.1 MB/s | 3.4 MB 00:00 2024-03-28T01:03:53.418 INFO:teuthology.orchestra.run.smithi033.stdout:(153/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 5.1 MB/s | 3.1 MB 00:00 2024-03-28T01:03:53.602 INFO:teuthology.orchestra.run.smithi033.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 230 kB/s | 42 kB 00:00 2024-03-28T01:03:53.669 INFO:teuthology.orchestra.run.smithi033.stdout:(155/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.9 MB/s | 2.3 MB 00:00 2024-03-28T01:03:53.736 INFO:teuthology.orchestra.run.smithi033.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 71 kB 00:00 2024-03-28T01:03:53.761 INFO:teuthology.orchestra.run.smithi033.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.5 MB/s | 248 kB 00:00 2024-03-28T01:03:53.795 INFO:teuthology.orchestra.run.smithi033.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 627 kB/s | 36 kB 00:00 2024-03-28T01:03:53.802 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T01:03:53.858 INFO:teuthology.orchestra.run.smithi033.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-28T01:03:53.891 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T01:03:53.896 INFO:teuthology.orchestra.run.smithi033.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-28T01:03:53.921 INFO:teuthology.orchestra.run.smithi033.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 372 kB/s | 23 kB 00:00 2024-03-28T01:03:53.988 INFO:teuthology.orchestra.run.smithi033.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 108 kB 00:00 2024-03-28T01:03:54.022 INFO:teuthology.orchestra.run.smithi033.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-28T01:03:54.033 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T01:03:54.055 INFO:teuthology.orchestra.run.smithi033.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-28T01:03:54.072 INFO:teuthology.orchestra.run.smithi033.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-03-28T01:03:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T01:03:54.106 INFO:teuthology.orchestra.run.smithi033.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 579 kB/s | 29 kB 00:00 2024-03-28T01:03:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T01:03:54.165 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T01:03:54.221 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T01:03:54.223 INFO:teuthology.orchestra.run.smithi033.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 8.1 MB/s | 1.2 MB 00:00 2024-03-28T01:03:54.274 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T01:03:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:(168/174): NetworkManager-libnm-1.47.2-1.el9.x8 1.8 MB/s | 1.8 MB 00:00 2024-03-28T01:03:54.316 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T01:03:54.382 INFO:teuthology.orchestra.run.smithi033.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.8 MB/s | 2.2 MB 00:00 2024-03-28T01:03:54.466 INFO:teuthology.orchestra.run.smithi033.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.5 MB/s | 2.3 MB 00:00 2024-03-28T01:03:54.491 INFO:teuthology.orchestra.run.smithi033.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 289 kB/s | 31 kB 00:00 2024-03-28T01:03:54.533 INFO:teuthology.orchestra.run.smithi033.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 1.8 MB/s | 465 kB 00:00 2024-03-28T01:03:54.574 INFO:teuthology.orchestra.run.smithi033.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 899 kB/s | 74 kB 00:00 2024-03-28T01:03:54.616 INFO:teuthology.orchestra.run.smithi033.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.2 MB/s | 1.1 MB 00:00 2024-03-28T01:03:54.625 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-03-28T01:03:54.626 INFO:teuthology.orchestra.run.smithi033.stdout:Total 16 MB/s | 225 MB 00:14 2024-03-28T01:03:54.717 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T01:03:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T01:03:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T01:03:55.066 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T01:03:55.243 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T01:03:55.300 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T01:03:55.360 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T01:03:55.447 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T01:03:55.516 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T01:03:55.556 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T01:03:55.641 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T01:03:55.683 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T01:03:55.747 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T01:03:55.821 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T01:03:56.004 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T01:03:56.053 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T01:03:56.092 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T01:03:56.140 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T01:03:56.264 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T01:03:56.298 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T01:03:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T01:03:56.372 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T01:03:56.373 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-03-28T01:03:56.396 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T01:03:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T01:03:56.725 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T01:03:56.771 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-03-28T01:03:56.771 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-03-28T01:03:56.855 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T01:03:56.906 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T01:03:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T01:03:57.068 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T01:03:57.315 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T01:03:57.368 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T01:03:57.540 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T01:03:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T01:03:57.656 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T01:03:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T01:03:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T01:03:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T01:03:57.860 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T01:03:57.919 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T01:03:57.972 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T01:03:58.099 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T01:03:58.144 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T01:03:58.178 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T01:03:58.239 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T01:03:58.281 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T01:03:58.422 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T01:03:58.542 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T01:03:58.594 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T01:03:58.630 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T01:03:58.685 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T01:03:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T01:03:58.896 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T01:03:59.433 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-03-28T01:03:59.434 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-03-28T01:03:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T01:03:59.911 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T01:03:59.946 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T01:04:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T01:04:00.123 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T01:04:00.170 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T01:04:00.413 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T01:04:00.450 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T01:04:00.542 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T01:04:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T01:04:01.472 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T01:04:02.973 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T01:04:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T01:04:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T01:04:03.130 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T01:04:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T01:04:03.540 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T01:04:03.569 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T01:04:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T01:04:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T01:04:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T01:04:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T01:04:04.347 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T01:04:04.438 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T01:04:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T01:04:07.013 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T01:04:07.053 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T01:04:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-03-28T01:04:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T01:04:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T01:04:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T01:04:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T01:04:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T01:04:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T01:04:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T01:04:07.654 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T01:04:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T01:04:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T01:04:07.753 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T01:04:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T01:04:07.900 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T01:04:07.900 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T01:04:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:07.949 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T01:04:08.196 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T01:04:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T01:04:08.436 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T01:04:08.464 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T01:04:08.573 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T01:04:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T01:04:08.666 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T01:04:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T01:04:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T01:04:09.085 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T01:04:09.231 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T01:04:09.261 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T01:04:09.343 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T01:04:09.388 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T01:04:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T01:04:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T01:04:09.806 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T01:04:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T01:04:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T01:04:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T01:04:10.412 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T01:04:10.502 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T01:04:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T01:04:10.831 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T01:04:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T01:04:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T01:04:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T01:04:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T01:04:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T01:04:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T01:04:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T01:04:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T01:04:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T01:04:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T01:04:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T01:04:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T01:04:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T01:04:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T01:04:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T01:04:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T01:04:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T01:04:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T01:04:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T01:04:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T01:04:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T01:04:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T01:04:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T01:04:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T01:04:14.099 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T01:04:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T01:04:14.310 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T01:04:14.339 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T01:04:14.383 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T01:04:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T01:04:14.486 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T01:04:14.526 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T01:04:14.945 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T01:04:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T01:04:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T01:04:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T01:04:15.530 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T01:04:15.577 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T01:04:15.646 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T01:04:15.769 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T01:04:15.811 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T01:04:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T01:04:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T01:04:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T01:04:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T01:04:16.099 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T01:04:16.292 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T01:04:16.347 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T01:04:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T01:04:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T01:04:16.590 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T01:04:16.667 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T01:04:16.715 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T01:04:16.750 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T01:04:16.774 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T01:04:16.925 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T01:04:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T01:04:17.271 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T01:04:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T01:04:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T01:04:17.465 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T01:04:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T01:04:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T01:04:17.948 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T01:04:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T01:04:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T01:04:18.090 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T01:04:18.153 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T01:04:18.206 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T01:04:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T01:04:18.332 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T01:04:18.377 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T01:04:18.499 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T01:04:18.548 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T01:04:18.592 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T01:04:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T01:04:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T01:04:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T01:04:19.008 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T01:04:19.108 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T01:04:19.471 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T01:04:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T01:04:19.717 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T01:04:19.803 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T01:04:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T01:04:20.618 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2024-03-28T01:04:20.619 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2024-03-28T01:04:20.619 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2024-03-28T01:04:20.619 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2024-03-28T01:04:20.619 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2024-03-28T01:04:20.619 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /root 2024-03-28T01:04:20.619 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2024-03-28T01:04:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T01:04:20.823 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T01:04:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T01:04:20.857 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T01:04:20.882 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T01:04:20.883 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:20.883 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T01:04:20.883 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:20.906 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T01:04:21.019 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T01:04:21.057 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T01:04:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T01:04:21.337 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T01:04:21.435 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T01:04:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T01:04:22.403 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T01:04:22.471 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T01:04:22.547 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T01:04:22.547 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:22.547 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T01:04:22.547 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T01:04:22.547 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T01:04:22.547 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:22.796 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T01:04:22.864 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T01:04:22.864 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:22.864 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T01:04:22.864 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T01:04:22.864 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T01:04:22.864 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:23.339 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T01:04:23.407 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T01:04:23.407 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:23.407 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T01:04:23.407 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T01:04:23.407 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T01:04:23.407 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T01:04:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T01:04:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T01:04:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T01:04:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T01:04:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T01:04:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T01:04:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T01:04:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T01:04:24.341 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T01:04:24.370 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T01:04:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T01:04:24.679 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T01:04:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T01:04:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T01:04:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T01:04:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T01:04:24.895 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T01:04:25.438 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T01:04:25.467 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T01:04:25.559 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T01:04:25.587 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T01:04:25.790 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T01:04:25.821 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T01:04:25.961 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T01:04:26.133 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T01:04:26.165 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T01:04:26.233 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T01:04:26.233 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:26.233 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T01:04:26.233 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T01:04:26.233 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T01:04:26.233 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:26.290 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T01:04:26.324 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T01:04:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T01:04:27.400 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T01:04:27.434 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T01:04:27.519 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T01:04:27.519 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:27.519 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T01:04:27.519 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T01:04:27.519 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T01:04:27.519 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:27.568 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T01:04:27.636 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T01:04:27.636 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:27.636 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T01:04:27.637 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T01:04:27.792 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T01:04:27.792 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:27.792 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T01:04:27.792 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T01:04:27.792 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T01:04:27.792 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:28.153 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T01:04:28.218 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T01:04:28.219 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:28.219 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T01:04:28.219 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T01:04:28.219 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T01:04:28.219 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T01:04:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T01:04:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T01:04:29.137 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T01:04:29.291 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T01:04:29.291 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T01:04:29.291 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T01:04:29.291 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:29.330 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T01:04:33.001 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T01:04:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T01:04:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T01:04:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T01:04:33.437 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T01:04:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T01:04:33.615 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T01:04:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T01:04:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T01:04:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T01:04:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T01:04:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T01:04:33.952 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T01:04:34.040 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T01:04:34.142 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T01:04:34.204 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T01:04:35.083 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T01:04:35.131 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T01:04:35.183 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T01:04:36.447 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T01:04:36.535 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T01:04:36.591 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T01:04:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T01:04:36.762 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T01:04:36.795 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T01:04:36.830 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T01:04:36.871 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T01:04:36.871 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T01:04:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T01:04:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T01:04:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T01:04:37.189 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T01:04:37.232 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T01:04:37.287 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T01:04:37.320 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T01:04:37.352 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T01:04:37.421 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T01:04:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T01:04:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T01:04:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T01:04:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T01:04:37.666 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T01:04:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T01:04:37.736 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T01:04:37.736 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T01:04:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T01:04:38.052 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T01:04:38.213 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T01:04:38.243 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T01:04:38.298 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T01:04:38.321 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T01:04:38.361 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T01:04:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T01:04:38.730 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T01:04:38.730 INFO:teuthology.orchestra.run.smithi194.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T01:04:38.730 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:43.406 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T01:04:43.406 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-03-28T01:04:43.406 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-03-28T01:04:43.406 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-03-28T01:04:43.406 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-03-28T01:04:43.407 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-03-28T01:04:43.407 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-03-28T01:04:43.407 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-03-28T01:04:43.407 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T01:04:43.557 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T01:04:43.558 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T01:04:43.559 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T01:04:43.559 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T01:04:43.559 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T01:04:43.559 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T01:04:43.559 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T01:04:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T01:04:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T01:04:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T01:04:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T01:04:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T01:04:43.580 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T01:04:43.581 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T01:04:43.582 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T01:04:43.583 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T01:04:43.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T01:04:43.585 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T01:04:43.586 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T01:04:43.587 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T01:04:43.588 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T01:04:43.589 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T01:04:43.590 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T01:04:43.591 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T01:04:43.591 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T01:04:43.591 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T01:04:43.591 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T01:04:43.676 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T01:04:43.728 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T01:04:43.728 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:43.728 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T01:04:43.728 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:45.395 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T01:04:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T01:04:45.484 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:45.484 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T01:04:45.484 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T01:04:45.484 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T01:04:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T01:04:45.818 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T01:04:45.818 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:45.818 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T01:04:45.818 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T01:04:45.818 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T01:04:45.818 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:46.064 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T01:04:46.064 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:46.064 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-03-28T01:04:46.064 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.065 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-03-28T01:04:46.066 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.067 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.068 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: jq-1.6-16.el9.x86_64 2024-03-28T01:04:46.069 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.070 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T01:04:46.071 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.072 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T01:04:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T01:04:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T01:04:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T01:04:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T01:04:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T01:04:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T01:04:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T01:04:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T01:04:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T01:04:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T01:04:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:04:46.078 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-03-28T01:04:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T01:04:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T01:04:46.396 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:46.396 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T01:04:46.396 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T01:04:46.396 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T01:04:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:46.514 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T01:04:46.722 DEBUG:teuthology.parallel:result is None 2024-03-28T01:04:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T01:04:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T01:04:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T01:04:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T01:04:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T01:04:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T01:04:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T01:04:48.932 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T01:04:48.961 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T01:04:49.111 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T01:04:49.292 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T01:04:49.322 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T01:04:49.415 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T01:04:49.415 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:49.415 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T01:04:49.415 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T01:04:49.415 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T01:04:49.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:49.481 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T01:04:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T01:04:49.631 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T01:04:50.670 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T01:04:50.700 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T01:04:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T01:04:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T01:04:50.770 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T01:04:50.770 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T01:04:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T01:04:50.896 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T01:04:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T01:04:50.896 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T01:04:51.108 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T01:04:51.108 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:51.108 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T01:04:51.108 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T01:04:51.108 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T01:04:51.108 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T01:04:51.619 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T01:04:51.619 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T01:04:51.620 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T01:04:51.620 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T01:04:51.620 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T01:04:51.620 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:04:56.718 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T01:04:56.772 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T01:04:56.807 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T01:04:56.845 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T01:04:57.153 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T01:04:57.286 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T01:04:57.335 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T01:04:57.407 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T01:04:57.467 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T01:04:57.512 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T01:04:57.539 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T01:04:57.583 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T01:04:57.636 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T01:04:57.733 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T01:04:57.838 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T01:04:57.885 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T01:04:58.818 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T01:04:58.864 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T01:04:58.918 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T01:05:00.301 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T01:05:00.386 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T01:05:00.428 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T01:05:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T01:05:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T01:05:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T01:05:00.665 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T01:05:00.706 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T01:05:00.707 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T01:05:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T01:05:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T01:05:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T01:05:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T01:05:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T01:05:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T01:05:01.188 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T01:05:01.221 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T01:05:01.291 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T01:05:01.377 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T01:05:01.434 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T01:05:01.475 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T01:05:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T01:05:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T01:05:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T01:05:01.615 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T01:05:01.615 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T01:05:01.960 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T01:05:01.960 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T01:05:02.107 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T01:05:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T01:05:02.184 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T01:05:02.223 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T01:05:02.286 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T01:05:02.387 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T01:05:02.657 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T01:05:02.658 INFO:teuthology.orchestra.run.smithi033.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T01:05:02.658 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T01:05:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T01:05:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T01:05:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T01:05:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T01:05:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T01:05:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T01:05:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T01:05:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T01:05:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T01:05:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T01:05:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T01:05:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T01:05:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T01:05:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T01:05:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T01:05:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T01:05:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T01:05:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T01:05:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T01:05:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T01:05:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T01:05:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T01:05:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T01:05:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T01:05:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T01:05:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.6-16.el9.x86_64 2024-03-28T01:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T01:05:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T01:05:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T01:05:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T01:05:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T01:05:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T01:05:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T01:05:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T01:05:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:10.180 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-03-28T01:05:10.858 DEBUG:teuthology.parallel:result is None 2024-03-28T01:05:10.859 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T01:05:11.079 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T01:05:11.139 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T01:05:11.139 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T01:05:11.139 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T01:05:11.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T01:05:11.288 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T01:05:11.360 INFO:teuthology.orchestra.run.smithi194.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T01:05:11.360 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T01:05:11.361 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T01:05:11.362 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-28T01:05:11.363 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:11.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T01:05:11.451 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:11.451 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T01:05:11.555 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-28T01:05:11.556 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:11.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T01:05:11.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T01:05:11.760 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:11.760 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T01:05:11.855 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T01:05:11.992 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-28T01:05:11.992 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:11.992 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T01:05:12.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T01:05:12.185 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:12.185 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T01:05:12.283 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T01:05:12.421 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-28T01:05:12.421 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:12.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T01:05:12.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T01:05:12.610 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:12.610 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T01:05:12.708 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T01:05:12.800 INFO:teuthology.run_tasks:Running task ceph... 2024-03-28T01:05:12.890 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-28T01:05:12.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2024-03-28T01:05:12.896 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /var/log/ceph 2024-03-28T01:05:12.991 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-28T01:05:12.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T01:05:12.995 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T01:05:13.092 INFO:tasks.ceph:Creating extra log directories... 2024-03-28T01:05:13.092 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T01:05:13.097 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T01:05:13.199 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-28T01:05:13.199 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-28T01:05:13.199 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-28T01:05:13.199 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:13.199 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-28T01:05:13.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-28T01:05:13.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-28T01:05:13.508 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:13.509 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-28T01:05:13.565 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-28T01:05:13.837 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-28T01:05:13.942 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-28T01:05:13.942 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-28T01:05:13.942 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252', 'branch': 'wip-mchangir-testing-main-20240318.032620', 'description': 'fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7625252', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing-main-20240318.032620'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing-main-20240318.032620', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}}, 'owner': 'mchangir', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 3645, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'sleep_before_teardown': 0, 'subset': '132/256', 'suite': 'fs', 'suite_branch': 'wip-mchangir-testing-main-20240318.032620', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdDFkbJ8neEy/kRcin/a5EEuG94zZpcsnyExwll9pO+AuwMq9VHSxbQV0SjsbKA7FlR9ml23QSJHNtvur+RG9Q=', 'smithi194.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAWl1fIfCuM29ayfT64fpl1N5+l5xGMs9311mNHxmwUSXAtKLdyj78swsnTh3QWOPEO/KcZnRNnde6WJJnQsBg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['client/test.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-27_03:45:23', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-28T01:05:13.943 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T01:05:13.948 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T01:05:14.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T01:05:14.033 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T01:05:14.131 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:14.132 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T01:05:14.177 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T01:05:14.177 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-03-28T01:05:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T01:05:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:14.257 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-28T01:05:14.257 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:14.257 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:14.257 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-03-28 01:05:03.126656123 +0000 2024-03-28T01:05:14.258 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-03-28 01:02:04.734969298 +0000 2024-03-28T01:05:14.258 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-03-28 01:02:04.734969298 +0000 2024-03-28T01:05:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-03-28 01:02:04.734969298 +0000 2024-03-28T01:05:14.258 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T01:05:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-03-28T01:05:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-03-28T01:05:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000381215 s, 1.3 MB/s 2024-03-28T01:05:14.367 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T01:05:14.406 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-03-28T01:05:14.484 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T01:05:14.484 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:14.484 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-28T01:05:14.484 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:14.484 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:14.484 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-03-28 01:05:03.126656123 +0000 2024-03-28T01:05:14.484 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-03-28 01:02:05.323959950 +0000 2024-03-28T01:05:14.485 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-03-28 01:02:05.323959950 +0000 2024-03-28T01:05:14.485 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-03-28 01:02:05.323959950 +0000 2024-03-28T01:05:14.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T01:05:14.590 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-03-28T01:05:14.590 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-03-28T01:05:14.590 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000277875 s, 1.8 MB/s 2024-03-28T01:05:14.592 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T01:05:14.631 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-03-28 01:05:03.127656107 +0000 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-03-28 01:02:05.925950396 +0000 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-03-28 01:02:05.925950396 +0000 2024-03-28T01:05:14.708 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-03-28 01:02:05.925950396 +0000 2024-03-28T01:05:14.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T01:05:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-03-28T01:05:14.834 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-03-28T01:05:14.834 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000271812 s, 1.9 MB/s 2024-03-28T01:05:14.838 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T01:05:14.893 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-03-28T01:05:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T01:05:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:14.939 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-28T01:05:14.939 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:14.939 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:14.939 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-03-28 01:05:03.128656091 +0000 2024-03-28T01:05:14.939 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-03-28 01:02:06.519940968 +0000 2024-03-28T01:05:14.940 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-03-28 01:02:06.519940968 +0000 2024-03-28T01:05:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-03-28 01:02:06.519940968 +0000 2024-03-28T01:05:14.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T01:05:15.070 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-03-28T01:05:15.070 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-03-28T01:05:15.070 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000356785 s, 1.4 MB/s 2024-03-28T01:05:15.075 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T01:05:15.135 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:15.135 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:15.135 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T01:05:15.190 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T01:05:15.191 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-28 01:04:39.192927956 +0000 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-28 01:01:39.339146424 +0000 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-28 01:01:39.339146424 +0000 2024-03-28T01:05:15.240 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-28 01:01:39.339146424 +0000 2024-03-28T01:05:15.241 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T01:05:15.322 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-28T01:05:15.322 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-28T01:05:15.322 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000356784 s, 1.4 MB/s 2024-03-28T01:05:15.325 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T01:05:15.379 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-28 01:04:39.193927936 +0000 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-28 01:01:39.945133947 +0000 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-28 01:01:39.945133947 +0000 2024-03-28T01:05:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-28 01:01:39.945133947 +0000 2024-03-28T01:05:15.428 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T01:05:15.512 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-28T01:05:15.512 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-28T01:05:15.512 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000368184 s, 1.4 MB/s 2024-03-28T01:05:15.516 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T01:05:15.573 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-03-28T01:05:15.623 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T01:05:15.623 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:15.623 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-28T01:05:15.623 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:15.623 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:15.624 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-28 01:04:39.194927915 +0000 2024-03-28T01:05:15.624 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-28 01:01:40.531121923 +0000 2024-03-28T01:05:15.624 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-28 01:01:40.531121923 +0000 2024-03-28T01:05:15.624 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-28 01:01:40.531121923 +0000 2024-03-28T01:05:15.624 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T01:05:15.702 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-28T01:05:15.702 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-28T01:05:15.702 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000317397 s, 1.6 MB/s 2024-03-28T01:05:15.705 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T01:05:15.758 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-03-28T01:05:15.805 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T01:05:15.805 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T01:05:15.805 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-28T01:05:15.805 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T01:05:15.806 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T01:05:15.806 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-28 01:04:39.194927915 +0000 2024-03-28T01:05:15.806 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-28 01:01:41.102110207 +0000 2024-03-28T01:05:15.806 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-28 01:01:41.102110207 +0000 2024-03-28T01:05:15.806 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-28 01:01:41.102110207 +0000 2024-03-28T01:05:15.806 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T01:05:15.893 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-28T01:05:15.894 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-28T01:05:15.894 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000472096 s, 1.1 MB/s 2024-03-28T01:05:15.897 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T01:05:15.953 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:15.953 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi194.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-28T01:05:15.953 INFO:tasks.ceph:Generating config... 2024-03-28T01:05:15.954 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-28T01:05:15.955 INFO:tasks.ceph:[client] debug client = 20 2024-03-28T01:05:15.955 INFO:tasks.ceph:[client] debug ms = 1 2024-03-28T01:05:15.955 INFO:tasks.ceph:[client] pid file = 2024-03-28T01:05:15.955 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-28T01:05:15.955 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-28T01:05:15.955 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-28T01:05:15.956 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-28T01:05:15.956 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-28T01:05:15.957 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-28T01:05:15.957 INFO:tasks.ceph:Setting up mon.a... 2024-03-28T01:05:15.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-28T01:05:16.109 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2024-03-28T01:05:16.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-28T01:05:16.265 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T01:05:16.366 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.b', '172.21.15.194'), ('mon.c', '[v2:172.21.15.194:3301,v1:172.21.15.194:6790]')] 2024-03-28T01:05:16.367 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.33,172.21.15.194,[v2:172.21.15.194:3301,v1:172.21.15.194:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-28T01:05:16.368 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:16.368 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T01:05:16.424 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.33 --add b 172.21.15.194 --addv c '[v2:172.21.15.194:3301,v1:172.21.15.194:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid 6560509b-7594-459b-89e1-d55d3bb3e1bc 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 6560509b-7594-459b-89e1-d55d3bb3e1bc 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2024-03-28T01:05:16.583109+0000 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-03-28T01:05:16.583109+0000 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 17 (quincy) 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:election_strategy: 1 2024-03-28T01:05:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-03-28T01:05:16.586 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.b 2024-03-28T01:05:16.586 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] mon.c 2024-03-28T01:05:16.586 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-28T01:05:16.588 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T01:05:16.639 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6560509b-7594-459b-89e1-d55d3bb3e1bc... 2024-03-28T01:05:16.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T01:05:16.647 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T01:05:16.814 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-03-28T01:05:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2024-03-28T01:05:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T01:05:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2024-03-28T01:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.33,172.21.15.194,[v2:172.21.15.194:3301,v1:172.21.15.194:6790]" 2024-03-28T01:05:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 6560509b-7594-459b-89e1-d55d3bb3e1bc 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2024-03-28T01:05:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug trim objects = true 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2024-03-28T01:05:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: bdev async discard = True 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: bdev enable discard = True 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore allocator = bitmap 2024-03-28T01:05:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 1/20 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 1/20 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 4/10 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.821 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2024-03-28T01:05:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T01:05:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: mon op complaint time = 120 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: client mount timeout = 600 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2024-03-28T01:05:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = "" 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout:[mds] 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds = 20 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds balancer = 20 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug frag = True 2024-03-28T01:05:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug scatterstat = True 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: mds op complaint time = 180 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: mds verify scatter = True 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi194.stdout:[global] 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi194.stdout: chdir = "" 2024-03-28T01:05:16.826 INFO:teuthology.orchestra.run.smithi194.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: auth supported = cephx 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: filestore xattr use omap = true 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: mon clock drift allowed = 1.000 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: osd crush chooseleaf type = 0 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: auth debug = true 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: ms die on old message = true 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: ms die on bug = true 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T01:05:16.827 INFO:teuthology.orchestra.run.smithi194.stdout: mon pg warn max object skew = 0 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: osd pool default size = 2 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd allow primary affinity = true 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd allow pg remap = true 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on legacy crush tunables = false 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on crush straw calc version zero = false 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on no sortbitwise = false 2024-03-28T01:05:16.828 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on osd down out interval zero = false 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on too few osds = false 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: mon_allow_pool_size_one = true 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: osd default data pool replay window = 5 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: mon allow pool delete = true 2024-03-28T01:05:16.829 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: mon cluster log file level = debug 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: debug asserts on shutdown = true 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: mon health detail to clog = false 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: mon host = "172.21.15.33,172.21.15.194,[v2:172.21.15.194:3301,v1:172.21.15.194:6790]" 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: fsid = 6560509b-7594-459b-89e1-d55d3bb3e1bc 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout:[osd] 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: osd journal size = 100 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: osd scrub load threshold = 5.0 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: osd scrub max interval = 600 2024-03-28T01:05:16.830 INFO:teuthology.orchestra.run.smithi194.stdout: osd mclock profile = high_recovery_ops 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd recover clone overlap = true 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd recovery max chunk = 1048576 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug shutdown = true 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug op order = true 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug verify stray on activate = true 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug trim objects = true 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd open classes on start = true 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug pg log writeout = true 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd deep scrub update digest min age = 30 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.831 INFO:teuthology.orchestra.run.smithi194.stdout: osd map max advance = 10 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: journal zero on create = true 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: filestore ondisk finisher threads = 3 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: filestore apply finisher threads = 3 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: bdev debug aio = true 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug misdirected ops = true 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: bdev async discard = True 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: bdev enable discard = True 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore allocator = bitmap 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore block size = 96636764160 2024-03-28T01:05:16.832 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore fsck on mount = True 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: debug bluefs = 1/20 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: debug bluestore = 1/20 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: debug osd = 20 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: debug rocksdb = 4/10 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd full ratio = 0.9 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd nearfull ratio = 0.8 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: osd failsafe full ratio = 0.95 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: osd objectstore = bluestore 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: osd op complaint time = 180 2024-03-28T01:05:16.833 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout:[mgr] 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: debug mgr = 20 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: debug mon = 20 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: debug auth = 20 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min pgs per osd = 4 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min bytes per osd = 10 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: mgr/telemetry/nag = false 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout:[mon] 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: debug mon = 20 2024-03-28T01:05:16.834 INFO:teuthology.orchestra.run.smithi194.stdout: debug paxos = 20 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: debug auth = 20 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: mon data avail warn = 5 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: mon mgr mkfs grace = 240 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min pgs per osd = 4 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd reporter subtree level = osd 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd prime pg temp = true 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min bytes per osd = 10 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.835 INFO:teuthology.orchestra.run.smithi194.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: # 1m isn't quite enough 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: mon_down_mkfs_grace = 2m 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_filestore_osds = false 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: mon op complaint time = 120 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout:[client] 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: rgw cache enabled = true 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: rgw enable ops log = true 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: rgw enable usage log = true 2024-03-28T01:05:16.836 INFO:teuthology.orchestra.run.smithi194.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: client mount timeout = 600 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: debug client = 20 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: pid file = "" 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: rados mon op timeout = 900 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: rados osd op timeout = 900 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.a] 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.b] 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.c] 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.a] 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.b] 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout:[mds] 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: debug mds = 20 2024-03-28T01:05:16.837 INFO:teuthology.orchestra.run.smithi194.stdout: debug mds balancer = 20 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: mds debug frag = True 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: mds debug scatterstat = True 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: mds op complaint time = 180 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: mds verify scatter = True 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: osd op complaint time = 180 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: rados mon op timeout = 900 2024-03-28T01:05:16.838 INFO:teuthology.orchestra.run.smithi194.stdout: rados osd op timeout = 900 2024-03-28T01:05:16.866 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-28T01:05:16.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-28T01:05:17.023 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-28T01:05:17.023 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:17.023 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-28T01:05:17.080 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:17.080 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-28T01:05:17.131 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:05:17.131 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:17.131 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T01:05:17.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T01:05:17.288 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:17.288 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:17.389 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:05:17.389 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:17.389 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T01:05:17.389 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T01:05:17.543 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:17.544 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:17.639 INFO:tasks.ceph:Setting up mon nodes... 2024-03-28T01:05:17.639 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-28T01:05:17.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-28T01:05:17.837 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-28T01:05:17.848 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T01:05:18.041 INFO:teuthology.orchestra.run.smithi194.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T01:05:18.053 INFO:tasks.ceph:Setting up mds nodes... 2024-03-28T01:05:18.053 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-28T01:05:18.251 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-28T01:05:18.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-28T01:05:18.367 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-28T01:05:18.551 INFO:teuthology.orchestra.run.smithi194.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-28T01:05:18.561 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-28T01:05:18.653 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-28T01:05:18.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-28T01:05:18.791 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-28T01:05:18.851 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-28T01:05:18.851 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi194.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-28T01:05:18.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-28T01:05:18.955 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:18.955 INFO:tasks.ceph:role: osd.0 2024-03-28T01:05:18.955 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:05:18.956 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:19.059 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-03-28T01:05:19.588 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-03-28T01:05:19.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-28T01:05:19.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-28T01:05:19.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-28T01:05:19.980 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:19.980 INFO:tasks.ceph:role: osd.1 2024-03-28T01:05:19.980 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:05:19.981 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-28T01:05:20.076 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:20.076 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:20.077 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:20.562 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-03-28T01:05:20.569 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-03-28T01:05:20.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-28T01:05:20.701 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-28T01:05:20.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-28T01:05:20.863 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:20.863 INFO:tasks.ceph:role: osd.2 2024-03-28T01:05:20.863 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:05:20.863 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-28T01:05:20.947 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:20.947 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:20.947 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:20.947 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:20.947 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:20.947 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:20.948 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:20.948 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:20.948 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:20.948 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:21.446 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-03-28T01:05:21.452 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-03-28T01:05:21.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-28T01:05:21.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-28T01:05:21.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-28T01:05:21.836 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:21.836 INFO:tasks.ceph:role: osd.3 2024-03-28T01:05:21.836 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:05:21.836 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-28T01:05:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:22.399 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-03-28T01:05:22.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-03-28T01:05:22.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-28T01:05:22.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-28T01:05:22.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:22.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:22.801+0000 7f6a2f321640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-28T01:05:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:22.804+0000 7f6a2f321640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-28T01:05:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:22.804+0000 7f6a2f321640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T01:05:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:22.804+0000 7f6a2f321640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T01:05:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:22.804+0000 7f6a2f321640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T01:05:22.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:22.809+0000 7f6a2f321640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-28T01:05:25.747 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-28T01:05:25.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:25.997+0000 7f6e12c0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-28T01:05:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:25.997+0000 7f6e12c0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-28T01:05:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:25.997+0000 7f6e12c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T01:05:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:25.997+0000 7f6e12c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T01:05:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:25.997+0000 7f6e12c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T01:05:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:26.000+0000 7f6e12c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-28T01:05:28.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-28T01:05:29.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:29.170+0000 7fc11f528640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-28T01:05:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:29.170+0000 7fc11f528640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-28T01:05:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:29.170+0000 7fc11f528640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T01:05:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:29.170+0000 7fc11f528640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T01:05:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:29.170+0000 7fc11f528640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T01:05:29.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:29.173+0000 7fc11f528640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-28T01:05:32.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-28T01:05:32.461 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:32.604+0000 7f8b2721f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-28T01:05:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:32.605+0000 7f8b2721f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-28T01:05:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:32.605+0000 7f8b2721f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T01:05:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:32.605+0000 7f8b2721f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T01:05:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:32.605+0000 7f8b2721f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T01:05:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:32.607+0000 7f8b2721f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-28T01:05:35.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-28T01:05:35.883 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-28T01:05:35.984 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:35.985 INFO:tasks.ceph:role: osd.4 2024-03-28T01:05:35.985 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:05:35.985 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:36.089 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:36.600 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-03-28T01:05:36.605 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-03-28T01:05:36.605 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-28T01:05:36.778 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-28T01:05:36.868 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-28T01:05:36.946 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:36.946 INFO:tasks.ceph:role: osd.5 2024-03-28T01:05:36.947 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:05:36.947 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-28T01:05:37.033 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:37.034 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:37.490 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-03-28T01:05:37.497 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-03-28T01:05:37.497 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-28T01:05:37.615 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-28T01:05:37.694 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-28T01:05:37.780 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:37.780 INFO:tasks.ceph:role: osd.6 2024-03-28T01:05:37.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:05:37.780 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-28T01:05:37.875 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:37.875 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:37.875 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:37.876 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:37.876 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:38.396 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-03-28T01:05:38.403 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-03-28T01:05:38.403 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-28T01:05:38.594 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-28T01:05:38.701 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-28T01:05:38.796 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T01:05:38.796 INFO:tasks.ceph:role: osd.7 2024-03-28T01:05:38.796 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:05:38.796 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-28T01:05:38.892 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T01:05:38.893 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T01:05:39.358 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-03-28T01:05:39.366 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-03-28T01:05:39.366 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-28T01:05:39.518 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-28T01:05:39.621 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:39.767 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:39.764+0000 7f66a3311640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-28T01:05:39.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:39.768+0000 7f66a3311640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-28T01:05:39.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:39.768+0000 7f66a3311640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T01:05:39.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:39.769+0000 7f66a3311640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T01:05:39.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:39.769+0000 7f66a3311640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T01:05:39.784 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:39.781+0000 7f66a3311640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-28T01:05:42.970 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-28T01:05:43.070 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:43.207 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:43.204+0000 7f727416b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-28T01:05:43.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:43.205+0000 7f727416b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-28T01:05:43.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:43.205+0000 7f727416b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T01:05:43.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:43.205+0000 7f727416b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T01:05:43.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:43.205+0000 7f727416b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T01:05:43.210 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:43.207+0000 7f727416b640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-28T01:05:43.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:05:43.951 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:05:46.131 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-28T01:05:46.224 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:46.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:46.355+0000 7f97bbf24640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-28T01:05:46.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:46.356+0000 7f97bbf24640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-28T01:05:46.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:46.356+0000 7f97bbf24640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T01:05:46.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:46.356+0000 7f97bbf24640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T01:05:46.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:46.356+0000 7f97bbf24640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T01:05:46.361 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:46.358+0000 7f97bbf24640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-28T01:05:49.280 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-28T01:05:49.377 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:49.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:49.505+0000 7f740b619640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-28T01:05:49.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:49.506+0000 7f740b619640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-28T01:05:49.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:49.506+0000 7f740b619640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T01:05:49.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:49.506+0000 7f740b619640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T01:05:49.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:49.506+0000 7f740b619640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T01:05:49.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:49.508+0000 7f740b619640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-28T01:05:52.444 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-28T01:05:52.535 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-28T01:05:52.535 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:52.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-28T01:05:52.638 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:52.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-28T01:05:52.737 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:52.737 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-28T01:05:52.839 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:52.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-28T01:05:52.941 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:52.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-28T01:05:53.042 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:53.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-28T01:05:53.141 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:53.141 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-28T01:05:53.228 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:53.228 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-28T01:05:53.320 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:53.320 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-28T01:05:53.448 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:53.448 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-28T01:05:53.534 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:53.534 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-28T01:05:53.669 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:53.669 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-28T01:05:53.762 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:53.762 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-28T01:05:53.823 INFO:tasks.ceph:Adding keys to all mons... 2024-03-28T01:05:53.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-28T01:05:53.830 DEBUG:teuthology.orchestra.run.smithi194:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-28T01:05:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2024-03-28T01:05:53.922 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBNwgRm/K/HMRAA4lyFiT94MKUtSVsyU42yqg== 2024-03-28T01:05:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-03-28T01:05:53.922 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBOwgRmk2bbDhAA4OB22ZXHMAOQW1ouNGTRlQ== 2024-03-28T01:05:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2024-03-28T01:05:53.922 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBSwgRmnb7JLxAAZYjgV8ic+OJkGhbL/ymM2Q== 2024-03-28T01:05:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBVwgRmLuh5OxAAJ0SP6YHG8dTA66cRqaI9BQ== 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBZwgRmjHguChAAeDG48sh8+EhNTPtkr+tM7A== 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBcwgRmLP8UJBAA7gjTSXO6izICjJxPy14zBQ== 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBOwgRm2BxXAhAA2wNjam2gfBpHfNzQc3dTEw== 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBOwgRmL3XKIBAAJh/jO48SS3wI1fK+HN6WsQ== 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBjwgRm70GhLRAAmFDmmZbWiF+s/QKSwHB+gg== 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2024-03-28T01:05:53.923 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBnwgRmG+dFDBAAfv4Ob3hokTIAcJDXIwAiiw== 2024-03-28T01:05:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2024-03-28T01:05:53.924 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBqwgRmpjJMFRAAxxvxJZyr/KkIteReSBSIzA== 2024-03-28T01:05:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2024-03-28T01:05:53.924 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBtwgRmnG44HhAABBNms3UBGWDZ5+7YmG3EHg== 2024-03-28T01:05:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-03-28T01:05:53.924 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBOwgRmC64ULxAACu8izpgZ6Jm6kU6vjdfnWw== 2024-03-28T01:05:53.924 INFO:teuthology.orchestra.run.smithi194.stdout:[mgr.y] 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBNwgRm/K/HMRAA4lyFiT94MKUtSVsyU42yqg== 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.a] 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBOwgRmk2bbDhAA4OB22ZXHMAOQW1ouNGTRlQ== 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.0] 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBSwgRmnb7JLxAAZYjgV8ic+OJkGhbL/ymM2Q== 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.1] 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBVwgRmLuh5OxAAJ0SP6YHG8dTA66cRqaI9BQ== 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.2] 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBZwgRmjHguChAAeDG48sh8+EhNTPtkr+tM7A== 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.3] 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBcwgRmLP8UJBAA7gjTSXO6izICjJxPy14zBQ== 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout:[mgr.x] 2024-03-28T01:05:53.925 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBOwgRm2BxXAhAA2wNjam2gfBpHfNzQc3dTEw== 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.b] 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBOwgRmL3XKIBAAJh/jO48SS3wI1fK+HN6WsQ== 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.4] 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBjwgRm70GhLRAAmFDmmZbWiF+s/QKSwHB+gg== 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.5] 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBnwgRmG+dFDBAAfv4Ob3hokTIAcJDXIwAiiw== 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.6] 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBqwgRmpjJMFRAAxxvxJZyr/KkIteReSBSIzA== 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.7] 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBtwgRmnG44HhAABBNms3UBGWDZ5+7YmG3EHg== 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout:[client.0] 2024-03-28T01:05:53.926 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQBOwgRmC64ULxAACu8izpgZ6Jm6kU6vjdfnWw== 2024-03-28T01:05:53.928 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T01:05:53.933 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T01:05:54.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T01:05:54.083 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T01:05:54.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.237 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.371 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.376 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.527 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.675 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.680 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:54.822 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T01:05:54.827 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T01:05:54.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T01:05:54.978 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T01:05:55.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.130 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.272 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.424 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.574 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T01:05:55.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T01:05:55.724 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T01:05:55.868 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-28T01:05:55.868 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-28T01:05:55.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T01:05:56.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-28T01:05:56.231 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-28T01:05:56.332 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T01:05:56.486 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-28T01:05:56.574 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-28T01:05:56.668 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T01:05:56.818 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-28T01:05:56.892 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:56.900 DEBUG:teuthology.orchestra.run.smithi194:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-28T01:05:56.951 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-28T01:05:56.952 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-28T01:05:56.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-28T01:05:56.959 INFO:tasks.ceph.mon.a:Started 2024-03-28T01:05:56.959 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-28T01:05:56.959 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-28T01:05:56.964 INFO:tasks.ceph.mon.b:Started 2024-03-28T01:05:56.964 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-28T01:05:56.965 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-28T01:05:56.970 INFO:tasks.ceph.mon.c:Started 2024-03-28T01:05:56.970 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-28T01:05:56.970 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-28T01:05:56.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-28T01:05:56.977 INFO:tasks.ceph.mgr.y:Started 2024-03-28T01:05:56.977 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-28T01:05:56.977 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-28T01:05:56.981 INFO:tasks.ceph.mgr.x:Started 2024-03-28T01:05:56.981 INFO:tasks.ceph:Setting crush tunables to default 2024-03-28T01:05:56.981 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2024-03-28T01:05:57.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.157+0000 7f528d200640 1 Processor -- start 2024-03-28T01:05:57.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.157+0000 7f528d200640 1 -- start start 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.158+0000 7f528d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.158+0000 7f528d200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 0x7f52881452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.158+0000 7f528d200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 0x7f5288148680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.158+0000 7f528d200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5288070590 con 0x7f52880b4aa0 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.158+0000 7f528d200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5288145c60 con 0x7f52881457f0 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.158+0000 7f528d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52880b2570 con 0x7f528806d370 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f527d400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 0x7f52881452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5286800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 0x7f5288148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f527d400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 0x7f52881452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55374/0 (socket says 172.21.15.33:55374) 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f527d400640 1 -- 172.21.15.33:0/1734516322 learned_addr learned my addr 172.21.15.33:0/1734516322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5285e00640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f527d400640 1 -- 172.21.15.33:0/1734516322 >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 msgr2=0x7f52881452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f527d400640 1 -- 172.21.15.33:0/1734516322 >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 msgr2=0x7f52881452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5286800640 1 -- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 msgr2=0x7f5288148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f527d400640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 0x7f52881452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5286800640 1 -- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 msgr2=0x7f5288148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5285e00640 1 -- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 msgr2=0x7f52880b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5286800640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 0x7f5288148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5285e00640 1 -- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 msgr2=0x7f52880b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5285e00640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5285e00640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f5286800640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 0x7f5288148680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T01:05:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.159+0000 7f527d400640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 0x7f52881452b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T01:05:57.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.360+0000 7f527d400640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.33:3300/0 conn(0x7f52880b4aa0 0x7f52881452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.360+0000 7f5285e00640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.360+0000 7f5286800640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 0x7f5288148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.361+0000 7f527d400640 1 -- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 msgr2=0x7f52880b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.361+0000 7f527d400640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.361+0000 7f527d400640 1 -- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 msgr2=0x7f5288148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.361+0000 7f527d400640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 0x7f5288148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.361+0000 7f527d400640 1 -- 172.21.15.33:0/1734516322 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528806d750 con 0x7f52880b4aa0 2024-03-28T01:05:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.361+0000 7f5285e00640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:05:57.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.363+0000 7f527d400640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 0x7f52881452b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f528806f2c0 tx=0x7f527002bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea983016309ceb21 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:57.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.364+0000 7f5285400640 1 -- 172.21.15.33:0/1734516322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270039070 con 0x7f52880b4aa0 2024-03-28T01:05:57.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.364+0000 7f5285400640 1 -- 172.21.15.33:0/1734516322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527002c980 con 0x7f52880b4aa0 2024-03-28T01:05:57.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.364+0000 7f5285400640 1 -- 172.21.15.33:0/1734516322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000f040 con 0x7f52880b4aa0 2024-03-28T01:05:57.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.365+0000 7f528d200640 1 -- 172.21.15.33:0/1734516322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 msgr2=0x7f52881452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:57.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.365+0000 7f528d200640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 0x7f52881452b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f528806f2c0 tx=0x7f527002bbd0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.365+0000 7f528d200640 1 -- 172.21.15.33:0/1734516322 shutdown_connections 2024-03-28T01:05:57.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.365+0000 7f528d200640 1 --2- 172.21.15.33:0/1734516322 >> v2:172.21.15.194:3300/0 conn(0x7f52881457f0 0x7f5288148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.365+0000 7f528d200640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 0x7f52881452b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.365+0000 7f528d200640 1 --2- 172.21.15.33:0/1734516322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.365+0000 7f528d200640 1 -- 172.21.15.33:0/1734516322 >> 172.21.15.33:0/1734516322 conn(0x7f52880b5320 msgr2=0x7f52880b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:57.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.366+0000 7f528d200640 1 -- 172.21.15.33:0/1734516322 shutdown_connections 2024-03-28T01:05:57.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.366+0000 7f528d200640 1 -- 172.21.15.33:0/1734516322 wait complete. 2024-03-28T01:05:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.369+0000 7f528d200640 1 Processor -- start 2024-03-28T01:05:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.369+0000 7f528d200640 1 -- start start 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.371+0000 7f528d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.371+0000 7f528d200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 0x7f5288142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.371+0000 7f528d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52881457f0 0x7f52880b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f528d200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52880af8c0 con 0x7f52880b4aa0 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f528d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52880b2810 con 0x7f52881457f0 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f528d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52880afa40 con 0x7f528806d370 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:47340/0 (socket says 172.21.15.33:47340) 2024-03-28T01:05:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 -- 172.21.15.33:0/3777164944 learned_addr learned my addr 172.21.15.33:0/3777164944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:05:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f527d400640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 0x7f5288142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5286800640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52881457f0 0x7f52880b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 -- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52881457f0 msgr2=0x7f52880b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52881457f0 0x7f52880b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 -- 172.21.15.33:0/3777164944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 msgr2=0x7f5288142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 0x7f5288142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.372+0000 7f5285e00640 1 -- 172.21.15.33:0/3777164944 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52880b0020 con 0x7f528806d370 2024-03-28T01:05:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.373+0000 7f5285e00640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b1710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f527800b6c0 tx=0x7f527800bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.373+0000 7f527ea00640 1 -- 172.21.15.33:0/3777164944 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278018070 con 0x7f528806d370 2024-03-28T01:05:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.373+0000 7f527ea00640 1 -- 172.21.15.33:0/3777164944 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5278010830 con 0x7f528806d370 2024-03-28T01:05:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.373+0000 7f527ea00640 1 -- 172.21.15.33:0/3777164944 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278010c20 con 0x7f528806d370 2024-03-28T01:05:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.373+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52880b0300 con 0x7f528806d370 2024-03-28T01:05:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.374+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528813b900 con 0x7f528806d370 2024-03-28T01:05:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.374+0000 7f527ea00640 1 -- 172.21.15.33:0/3777164944 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f527801b070 con 0x7f528806d370 2024-03-28T01:05:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.374+0000 7f527ea00640 1 -- 172.21.15.33:0/3777164944 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f52780147b0 con 0x7f528806d370 2024-03-28T01:05:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.375+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52880afe40 con 0x7f528806d370 2024-03-28T01:05:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.382+0000 7f527ea00640 1 -- 172.21.15.33:0/3777164944 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f52880afe40 con 0x7f528806d370 2024-03-28T01:05:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.457+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5288075310 con 0x7f528806d370 2024-03-28T01:05:57.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.464+0000 7f527ea00640 1 -- 172.21.15.33:0/3777164944 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f52780149b0 con 0x7f528806d370 2024-03-28T01:05:57.467 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-03-28T01:05:57.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.465+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 msgr2=0x7f52880b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.465+0000 7f528d200640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b1710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f527800b6c0 tx=0x7f527800bb80 comp rx=0 tx=0).stop 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.465+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 shutdown_connections 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.465+0000 7f528d200640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52881457f0 0x7f52880b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.465+0000 7f528d200640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52880b4aa0 0x7f5288142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.466+0000 7f528d200640 1 --2- 172.21.15.33:0/3777164944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f528806d370 0x7f52880b1710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.466+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 >> 172.21.15.33:0/3777164944 conn(0x7f52880b5320 msgr2=0x7f528806c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.466+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 shutdown_connections 2024-03-28T01:05:57.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:05:57.466+0000 7f528d200640 1 -- 172.21.15.33:0/3777164944 wait complete. 2024-03-28T01:05:57.486 INFO:tasks.ceph:check_enable_crimson: False 2024-03-28T01:05:57.487 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-28T01:05:57.487 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:57.487 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-28T01:05:57.566 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:57.566 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-28T01:05:57.697 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:57.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-28T01:05:57.765 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:57.762+0000 7f6b1563b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T01:05:57.829 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:05:57.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-28T01:05:57.834 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:57.830+0000 7f6b1563b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T01:05:57.926 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:57.926 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-28T01:05:57.938 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:57.936+0000 7f24ca22b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T01:05:58.007 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:58.005+0000 7f24ca22b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T01:05:58.021 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:58.021 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-28T01:05:58.151 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:58.151 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-28T01:05:58.273 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-28T01:05:58.274 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-28T01:05:58.404 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 2867531b-5f43-4ce2-82ef-35bfdfd9af3d 0 2024-03-28T01:05:58.498 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:58.494+0000 7f6b1563b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T01:05:58.565 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:58.562+0000 7f6b1563b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T01:05:58.619 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:58.615+0000 7f6b1563b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T01:05:58.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.638+0000 7fd209600640 1 Processor -- start 2024-03-28T01:05:58.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.638+0000 7fd209600640 1 -- start start 2024-03-28T01:05:58.642 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.639+0000 7fd209600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2040b9530 0x7fd2040b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.642 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.639+0000 7fd209600640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fd2040b4a30 0x7fd204147020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.642 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.639+0000 7fd209600640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2040b6300 con 0x7fd204140cb0 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.639+0000 7fd209600640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2040b6570 con 0x7fd2040b4a30 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.639+0000 7fd209600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040b5200 con 0x7fd2040b9530 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd1fb400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fd2040b4a30 0x7fd204147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd1fb400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fd2040b4a30 0x7fd204147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49272/0 (socket says 172.21.15.194:49272) 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd1fb400640 1 -- 172.21.15.194:0/3815916650 learned_addr learned my addr 172.21.15.194:0/3815916650 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd208c00640 1 --1- 172.21.15.194:0/3815916650 >> v1:172.21.15.194:6789/0 conn(0x7fd204140cb0 0x7fd2041410e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.194:59878/0 (socket says 172.21.15.194:59878) 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd203e00640 1 --2- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2040b9530 0x7fd2040b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd1fb400640 1 -- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2040b9530 msgr2=0x7fd2040b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd1fb400640 1 --2- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2040b9530 0x7fd2040b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd1fb400640 1 -- 172.21.15.194:0/3815916650 >> v1:172.21.15.194:6789/0 conn(0x7fd204140cb0 legacy=0x7fd2041410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.643 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.640+0000 7fd1fb400640 1 -- 172.21.15.194:0/3815916650 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040b4510 con 0x7fd2040b4a30 2024-03-28T01:05:58.644 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.641+0000 7fd1fb400640 1 --2- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204147020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009870 tx=0x7fd1ec02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da605179698c2a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:58.644 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd203400640 1 -- 172.21.15.194:0/3815916650 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec035020 con 0x7fd2040b4a30 2024-03-28T01:05:58.644 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd203400640 1 -- 172.21.15.194:0/3815916650 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec031870 con 0x7fd2040b4a30 2024-03-28T01:05:58.644 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd203400640 1 -- 172.21.15.194:0/3815916650 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031c40 con 0x7fd2040b4a30 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 -- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 msgr2=0x7fd204147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 --2- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204147020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009870 tx=0x7fd1ec02b910 comp rx=0 tx=0).stop 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 -- 172.21.15.194:0/3815916650 shutdown_connections 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 --2- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204147020 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 --2- 172.21.15.194:0/3815916650 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2040b9530 0x7fd2040b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 -- 172.21.15.194:0/3815916650 >> 172.21.15.194:0/3815916650 conn(0x7fd204072760 msgr2=0x7fd204072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 -- 172.21.15.194:0/3815916650 shutdown_connections 2024-03-28T01:05:58.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.642+0000 7fd209600640 1 -- 172.21.15.194:0/3815916650 wait complete. 2024-03-28T01:05:58.646 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.644+0000 7fd209600640 1 Processor -- start 2024-03-28T01:05:58.646 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:58.644+0000 7f24ca22b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T01:05:58.646 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.644+0000 7fd209600640 1 -- start start 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd209600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd209600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040b9530 0x7fd20414a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd209600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd204140cb0 0x7fd204150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd209600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204147560 con 0x7fd2040b9530 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd209600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040b4e90 con 0x7fd2040b4a30 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd209600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041519a0 con 0x7fd204140cb0 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204143b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49276/0 (socket says 172.21.15.194:49276) 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 -- 172.21.15.194:0/1129202464 learned_addr learned my addr 172.21.15.194:0/1129202464 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd208c00640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd204140cb0 0x7fd204150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 -- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd204140cb0 msgr2=0x7fd204150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.649 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd204140cb0 0x7fd204150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 -- 172.21.15.194:0/1129202464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040b9530 msgr2=0x7fd20414a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040b9530 0x7fd20414a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd203e00640 1 -- 172.21.15.194:0/1129202464 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20414af50 con 0x7fd2040b4a30 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.645+0000 7fd208c00640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd204140cb0 0x7fd204150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd203e00640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204143b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400efc0 tx=0x7fd1f400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd200c00640 1 -- 172.21.15.194:0/1129202464 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4018020 con 0x7fd2040b4a30 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd200c00640 1 -- 172.21.15.194:0/1129202464 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4014480 con 0x7fd2040b4a30 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd200c00640 1 -- 172.21.15.194:0/1129202464 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4014850 con 0x7fd2040b4a30 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20414c7c0 con 0x7fd2040b4a30 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20414c4e0 con 0x7fd2040b4a30 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd200c00640 1 -- 172.21.15.194:0/1129202464 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd1f401b020 con 0x7fd2040b4a30 2024-03-28T01:05:58.650 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.646+0000 7fd200c00640 1 -- 172.21.15.194:0/1129202464 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd1f40133e0 con 0x7fd2040b4a30 2024-03-28T01:05:58.651 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.647+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd204075310 con 0x7fd2040b4a30 2024-03-28T01:05:58.652 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.650+0000 7fd200c00640 1 -- 172.21.15.194:0/1129202464 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd1f4022020 con 0x7fd2040b4a30 2024-03-28T01:05:58.686 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:58.682+0000 7f6b1563b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T01:05:58.723 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:58.721+0000 7f24ca22b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T01:05:58.735 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.733+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2867531b-5f43-4ce2-82ef-35bfdfd9af3d", "id": 0} v 0) v1 -- 0x7fd2040b5fb0 con 0x7fd2040b4a30 2024-03-28T01:05:58.738 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:58.735+0000 7f6b1563b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T01:05:58.741 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.739+0000 7fd200c00640 1 -- 172.21.15.194:0/1129202464 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2867531b-5f43-4ce2-82ef-35bfdfd9af3d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd1f4010040 con 0x7fd2040b4a30 2024-03-28T01:05:58.742 INFO:teuthology.orchestra.run.smithi194.stdout:0 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 msgr2=0x7fd204143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204143b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400efc0 tx=0x7fd1f400c490 comp rx=0 tx=0).stop 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 shutdown_connections 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd204140cb0 0x7fd204150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040b9530 0x7fd20414a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 --2- 172.21.15.194:0/1129202464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2040b4a30 0x7fd204143b10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 >> 172.21.15.194:0/1129202464 conn(0x7fd204072760 msgr2=0x7fd2040b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 shutdown_connections 2024-03-28T01:05:58.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.741+0000 7fd209600640 1 -- 172.21.15.194:0/1129202464 wait complete. 2024-03-28T01:05:58.760 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new e5d41e15-7330-4018-9ba8-6972f2bf9eb7 1 2024-03-28T01:05:58.780 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:58.778+0000 7f24ca22b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T01:05:58.851 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:58.849+0000 7f24ca22b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T01:05:58.875 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:58.872+0000 7f6b1563b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T01:05:58.905 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:58.903+0000 7f24ca22b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T01:05:58.942 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.939+0000 7fbbbf200640 1 Processor -- start 2024-03-28T01:05:58.942 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.940+0000 7fbbbf200640 1 -- start start 2024-03-28T01:05:58.944 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbf200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbbb80b9530 0x7fbbb80b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.944 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbf200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b4a30 0x7fbbb8147020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.944 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbf200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fbbb8140cb0 0x7fbbb81410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.944 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbf200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb80b6570 con 0x7fbbb80b9530 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbf200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb80b6300 con 0x7fbbb8140cb0 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbf200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80b5200 con 0x7fbbb80b4a30 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbd400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b4a30 0x7fbbb8147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbe800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fbbb8140cb0 0x7fbbb81410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbe800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fbbb8140cb0 0x7fbbb81410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49280/0 (socket says 172.21.15.194:49280) 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbd400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b4a30 0x7fbbb8147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:40472/0 (socket says 172.21.15.194:40472) 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbe800640 1 -- 172.21.15.194:0/1175216791 learned_addr learned my addr 172.21.15.194:0/1175216791 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbde00640 1 --2- 172.21.15.194:0/1175216791 >> v2:172.21.15.33:3300/0 conn(0x7fbbb80b9530 0x7fbbb80b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbd400640 1 -- 172.21.15.194:0/1175216791 >> v2:172.21.15.194:3300/0 conn(0x7fbbb8140cb0 msgr2=0x7fbbb81410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbd400640 1 --2- 172.21.15.194:0/1175216791 >> v2:172.21.15.194:3300/0 conn(0x7fbbb8140cb0 0x7fbbb81410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbd400640 1 -- 172.21.15.194:0/1175216791 >> v2:172.21.15.33:3300/0 conn(0x7fbbb80b9530 msgr2=0x7fbbb80b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbd400640 1 --2- 172.21.15.194:0/1175216791 >> v2:172.21.15.33:3300/0 conn(0x7fbbb80b9530 0x7fbbb80b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.945 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbd400640 1 -- 172.21.15.194:0/1175216791 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb80b4510 con 0x7fbbb80b4a30 2024-03-28T01:05:58.946 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.941+0000 7fbbbe800640 1 --2- 172.21.15.194:0/1175216791 >> v2:172.21.15.194:3300/0 conn(0x7fbbb8140cb0 0x7fbbb81410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:05:58.947 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.942+0000 7fbbbd400640 1 --2- 172.21.15.194:0/1175216791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b4a30 0x7fbbb8147020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80b6f40 tx=0x7fbba802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e4e91f6f767f723 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:58.947 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbca00640 1 -- 172.21.15.194:0/1175216791 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8039070 con 0x7fbbb80b4a30 2024-03-28T01:05:58.947 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbca00640 1 -- 172.21.15.194:0/1175216791 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba80316f0 con 0x7fbbb80b4a30 2024-03-28T01:05:58.947 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbca00640 1 -- 172.21.15.194:0/1175216791 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8031ac0 con 0x7fbbb80b4a30 2024-03-28T01:05:58.947 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbf200640 1 -- 172.21.15.194:0/1175216791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b4a30 msgr2=0x7fbbb8147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.947 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbf200640 1 --2- 172.21.15.194:0/1175216791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b4a30 0x7fbbb8147020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80b6f40 tx=0x7fbba802b790 comp rx=0 tx=0).stop 2024-03-28T01:05:58.947 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbf200640 1 -- 172.21.15.194:0/1175216791 shutdown_connections 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbf200640 1 --2- 172.21.15.194:0/1175216791 >> v2:172.21.15.194:3300/0 conn(0x7fbbb8140cb0 0x7fbbb81410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbf200640 1 --2- 172.21.15.194:0/1175216791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b4a30 0x7fbbb8147020 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbf200640 1 --2- 172.21.15.194:0/1175216791 >> v2:172.21.15.33:3300/0 conn(0x7fbbb80b9530 0x7fbbb80b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.943+0000 7fbbbf200640 1 -- 172.21.15.194:0/1175216791 >> 172.21.15.194:0/1175216791 conn(0x7fbbb8072760 msgr2=0x7fbbb8072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.944+0000 7fbbbf200640 1 -- 172.21.15.194:0/1175216791 shutdown_connections 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.944+0000 7fbbbf200640 1 -- 172.21.15.194:0/1175216791 wait complete. 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.945+0000 7fbbbf200640 1 Processor -- start 2024-03-28T01:05:58.948 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.945+0000 7fbbbf200640 1 -- start start 2024-03-28T01:05:58.950 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbf200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80b4a30 0x7fbbb813d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.950 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbf200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b9530 0x7fbbb81cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.950 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbf200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbb8140cb0 0x7fbbb81d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbf200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8147560 con 0x7fbbb80b4a30 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbf200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80b4e90 con 0x7fbbb8140cb0 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbf200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8141620 con 0x7fbbb80b9530 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b9530 0x7fbbb81cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b9530 0x7fbbb81cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:40476/0 (socket says 172.21.15.194:40476) 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbe800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbb8140cb0 0x7fbbb81d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 -- 172.21.15.194:0/3079692382 learned_addr learned my addr 172.21.15.194:0/3079692382 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbde00640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80b4a30 0x7fbbb813d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 -- 172.21.15.194:0/3079692382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbb8140cb0 msgr2=0x7fbbb81d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbb8140cb0 0x7fbbb81d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 -- 172.21.15.194:0/3079692382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80b4a30 msgr2=0x7fbbb813d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80b4a30 0x7fbbb813d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.946+0000 7fbbbd400640 1 -- 172.21.15.194:0/3079692382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb813d5f0 con 0x7fbbb80b9530 2024-03-28T01:05:58.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.947+0000 7fbbbd400640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b9530 0x7fbbb81cd770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbba8037040 tx=0x7fbba80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.947+0000 7fbbb2000640 1 -- 172.21.15.194:0/3079692382 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8043070 con 0x7fbbb80b9530 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.947+0000 7fbbb2000640 1 -- 172.21.15.194:0/3079692382 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba80346a0 con 0x7fbbb80b9530 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.947+0000 7fbbb2000640 1 -- 172.21.15.194:0/3079692382 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8039040 con 0x7fbbb80b9530 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.947+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb81ce320 con 0x7fbbb80b9530 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.947+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb81ce600 con 0x7fbbb80b9530 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.947+0000 7fbbb2000640 1 -- 172.21.15.194:0/3079692382 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbba8046070 con 0x7fbbb80b9530 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.948+0000 7fbbb2000640 1 -- 172.21.15.194:0/3079692382 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fbba803e980 con 0x7fbbb80b9530 2024-03-28T01:05:58.952 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.948+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb8075310 con 0x7fbbb80b9530 2024-03-28T01:05:58.953 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:58.951+0000 7fbbb2000640 1 -- 172.21.15.194:0/3079692382 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbba80315b0 con 0x7fbbb80b9530 2024-03-28T01:05:58.960 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:58.956+0000 7f6b1563b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T01:05:59.017 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:59.013+0000 7f6b1563b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T01:05:59.031 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.029+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e5d41e15-7330-4018-9ba8-6972f2bf9eb7", "id": 1} v 0) v1 -- 0x7fbbb81cf870 con 0x7fbbb80b9530 2024-03-28T01:05:59.038 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.035+0000 7fbbb2000640 1 -- 172.21.15.194:0/3079692382 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5d41e15-7330-4018-9ba8-6972f2bf9eb7", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbba8034a90 con 0x7fbbb80b9530 2024-03-28T01:05:59.038 INFO:teuthology.orchestra.run.smithi194.stdout:1 2024-03-28T01:05:59.039 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b9530 msgr2=0x7fbbb81cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.039 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b9530 0x7fbbb81cd770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbba8037040 tx=0x7fbba80046c0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.039 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 shutdown_connections 2024-03-28T01:05:59.040 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbb8140cb0 0x7fbbb81d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.040 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbbb80b9530 0x7fbbb81cd770 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.040 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 --2- 172.21.15.194:0/3079692382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80b4a30 0x7fbbb813d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.040 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 >> 172.21.15.194:0/3079692382 conn(0x7fbbb8072760 msgr2=0x7fbbb80b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:59.040 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 shutdown_connections 2024-03-28T01:05:59.040 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.037+0000 7fbbbf200640 1 -- 172.21.15.194:0/3079692382 wait complete. 2024-03-28T01:05:59.047 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:59.045+0000 7f24ca22b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T01:05:59.057 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new db38d592-5f5e-41a4-96e3-ff85f508d242 2 2024-03-28T01:05:59.082 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:59.078+0000 7f6b1563b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T01:05:59.131 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:59.129+0000 7f24ca22b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T01:05:59.140 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:59.136+0000 7f6b1563b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T01:05:59.189 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:59.187+0000 7f24ca22b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T01:05:59.251 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:59.249+0000 7f24ca22b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T01:05:59.283 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.281+0000 7faccf200640 1 Processor -- start 2024-03-28T01:05:59.283 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.281+0000 7faccf200640 1 -- start start 2024-03-28T01:05:59.285 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.282+0000 7faccf200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.285 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccf200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd006c610 con 0x7facd006c180 2024-03-28T01:05:59.285 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccf200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd00729d0 con 0x7facd0141040 2024-03-28T01:05:59.285 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccf200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd0072cb0 con 0x7facd0071d60 2024-03-28T01:05:59.285 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccd400640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7facd0141040 0x7facd006ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.194:59884/0 (socket says 172.21.15.194:59884) 2024-03-28T01:05:59.285 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccd400640 1 -- 172.21.15.194:0/1853436741 learned_addr learned my addr 172.21.15.194:0/1853436741 (peer_addr_for_me v1:172.21.15.194:0/0) 2024-03-28T01:05:59.285 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccde00640 1 --2- 172.21.15.194:0/1853436741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.286 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccca00640 1 -- 172.21.15.194:0/1853436741 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266522728 0 0) 0x7facd00729d0 con 0x7facd0141040 2024-03-28T01:05:59.286 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccde00640 1 -- 172.21.15.194:0/1853436741 >> v1:172.21.15.194:6789/0 conn(0x7facd0141040 legacy=0x7facd006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.286 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccde00640 1 -- 172.21.15.194:0/1853436741 >> v1:172.21.15.33:6789/0 conn(0x7facd006c180 legacy=0x7facd0151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:59.287 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.283+0000 7faccde00640 1 -- 172.21.15.194:0/1853436741 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd006c880 con 0x7facd0071d60 2024-03-28T01:05:59.287 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.284+0000 7faccde00640 1 --2- 172.21.15.194:0/1853436741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0072140 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7facc00097d0 tx=0x7facc002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d136fe25a09acd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccca00640 1 -- 172.21.15.194:0/1853436741 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0039070 con 0x7facd0071d60 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccca00640 1 -- 172.21.15.194:0/1853436741 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc00316f0 con 0x7facd0071d60 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccca00640 1 -- 172.21.15.194:0/1853436741 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0031ac0 con 0x7facd0071d60 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccf200640 1 -- 172.21.15.194:0/1853436741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 msgr2=0x7facd0072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccf200640 1 --2- 172.21.15.194:0/1853436741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0072140 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7facc00097d0 tx=0x7facc002b790 comp rx=0 tx=0).stop 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccf200640 1 -- 172.21.15.194:0/1853436741 shutdown_connections 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccf200640 1 --2- 172.21.15.194:0/1853436741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0072140 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.288 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccf200640 1 -- 172.21.15.194:0/1853436741 >> 172.21.15.194:0/1853436741 conn(0x7facd00b9860 msgr2=0x7facd00b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:59.289 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccf200640 1 -- 172.21.15.194:0/1853436741 shutdown_connections 2024-03-28T01:05:59.289 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.285+0000 7faccf200640 1 -- 172.21.15.194:0/1853436741 wait complete. 2024-03-28T01:05:59.289 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.287+0000 7faccf200640 1 Processor -- start 2024-03-28T01:05:59.289 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.287+0000 7faccf200640 1 -- start start 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.287+0000 7faccf200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd006c180 0x7facd01479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccf200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccf200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facd0141040 0x7facd01dce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccf200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd0141620 con 0x7facd006c180 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccf200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd01414a0 con 0x7facd0141040 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccf200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd01417a0 con 0x7facd0071d60 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7facce800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facd0141040 0x7facd01dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:40500/0 (socket says 172.21.15.194:40500) 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7facce800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facd0141040 0x7facd01dce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49296/0 (socket says 172.21.15.194:49296) 2024-03-28T01:05:59.292 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 -- 172.21.15.194:0/2921317759 learned_addr learned my addr 172.21.15.194:0/2921317759 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccde00640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd006c180 0x7facd01479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 -- 172.21.15.194:0/2921317759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facd0141040 msgr2=0x7facd01dce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facd0141040 0x7facd01dce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 -- 172.21.15.194:0/2921317759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd006c180 msgr2=0x7facd01479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd006c180 0x7facd01479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccd400640 1 -- 172.21.15.194:0/2921317759 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00b0990 con 0x7facd0071d60 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.288+0000 7faccde00640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd006c180 0x7facd01479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7faccd400640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0140140 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7facbc00c4d0 tx=0x7facbc00c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7facc6000640 1 -- 172.21.15.194:0/2921317759 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc018070 con 0x7facd0071d60 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7facc6000640 1 -- 172.21.15.194:0/2921317759 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc007500 con 0x7facd0071d60 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7facc6000640 1 -- 172.21.15.194:0/2921317759 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc004a20 con 0x7facd0071d60 2024-03-28T01:05:59.293 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd01495f0 con 0x7facd0071d60 2024-03-28T01:05:59.294 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd01474a0 con 0x7facd0071d60 2024-03-28T01:05:59.294 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7facc6000640 1 -- 172.21.15.194:0/2921317759 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7facbc004bc0 con 0x7facd0071d60 2024-03-28T01:05:59.294 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.289+0000 7facc6000640 1 -- 172.21.15.194:0/2921317759 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7facbc012820 con 0x7facd0071d60 2024-03-28T01:05:59.294 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.290+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd01476a0 con 0x7facd0071d60 2024-03-28T01:05:59.295 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.293+0000 7facc6000640 1 -- 172.21.15.194:0/2921317759 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7facd01476a0 con 0x7facd0071d60 2024-03-28T01:05:59.310 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:59.308+0000 7f24ca22b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T01:05:59.368 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.366+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "db38d592-5f5e-41a4-96e3-ff85f508d242", "id": 2} v 0) v1 -- 0x7facd0075310 con 0x7facd0071d60 2024-03-28T01:05:59.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.374+0000 7facc6000640 1 -- 172.21.15.194:0/2921317759 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db38d592-5f5e-41a4-96e3-ff85f508d242", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7facbc012400 con 0x7facd0071d60 2024-03-28T01:05:59.391 INFO:teuthology.orchestra.run.smithi194.stdout:2 2024-03-28T01:05:59.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 msgr2=0x7facd0140140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0140140 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7facbc00c4d0 tx=0x7facbc00c990 comp rx=0 tx=0).stop 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 shutdown_connections 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facd0141040 0x7facd01dce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7facd0071d60 0x7facd0140140 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 --2- 172.21.15.194:0/2921317759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd006c180 0x7facd01479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 >> 172.21.15.194:0/2921317759 conn(0x7facd00b9860 msgr2=0x7facd013a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 shutdown_connections 2024-03-28T01:05:59.392 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.376+0000 7faccf200640 1 -- 172.21.15.194:0/2921317759 wait complete. 2024-03-28T01:05:59.400 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 68f5578e-d4a0-48fb-8719-b6436beb25ce 3 2024-03-28T01:05:59.616 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:59.613+0000 7f6b1563b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T01:05:59.628 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.626+0000 7fe549200640 1 Processor -- start 2024-03-28T01:05:59.628 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.626+0000 7fe549200640 1 -- start start 2024-03-28T01:05:59.630 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.627+0000 7fe549200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5440b6a00 0x7fe544147020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.630 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.627+0000 7fe549200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5440b6300 con 0x7fe5440b9530 2024-03-28T01:05:59.630 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe549200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5440b5200 con 0x7fe544140cb0 2024-03-28T01:05:59.630 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe549200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440b6570 con 0x7fe5440b6a00 2024-03-28T01:05:59.630 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe542a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5440b6a00 0x7fe544147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.630 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe543e00640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7fe544140cb0 0x7fe5441410d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.194:59896/0 (socket says 172.21.15.194:59896) 2024-03-28T01:05:59.630 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe542a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5440b6a00 0x7fe544147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:40512/0 (socket says 172.21.15.194:40512) 2024-03-28T01:05:59.631 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe542a00640 1 -- 172.21.15.194:0/2528369921 learned_addr learned my addr 172.21.15.194:0/2528369921 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:59.631 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe542a00640 1 -- 172.21.15.194:0/2528369921 >> v1:172.21.15.194:6789/0 conn(0x7fe544140cb0 legacy=0x7fe5441410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.631 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe542a00640 1 -- 172.21.15.194:0/2528369921 >> v1:172.21.15.33:6789/0 conn(0x7fe5440b9530 legacy=0x7fe5440b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.631 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe542000640 1 -- 172.21.15.194:0/2528369921 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595421427 0 0) 0x7fe5440b5200 con 0x7fe544140cb0 2024-03-28T01:05:59.631 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.628+0000 7fe542a00640 1 -- 172.21.15.194:0/2528369921 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5440b4510 con 0x7fe5440b6a00 2024-03-28T01:05:59.632 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.630+0000 7fe542a00640 1 --2- 172.21.15.194:0/2528369921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5440b6a00 0x7fe544147020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe5300098a0 tx=0x7fe53002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948fb43bfac73146 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:59.632 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.630+0000 7fe542000640 1 -- 172.21.15.194:0/2528369921 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530034070 con 0x7fe5440b6a00 2024-03-28T01:05:59.632 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.630+0000 7fe542000640 1 -- 172.21.15.194:0/2528369921 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530030840 con 0x7fe5440b6a00 2024-03-28T01:05:59.632 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.630+0000 7fe542000640 1 -- 172.21.15.194:0/2528369921 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530030c10 con 0x7fe5440b6a00 2024-03-28T01:05:59.633 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.631+0000 7fe549200640 1 -- 172.21.15.194:0/2528369921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5440b6a00 msgr2=0x7fe544147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.633 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.631+0000 7fe549200640 1 --2- 172.21.15.194:0/2528369921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5440b6a00 0x7fe544147020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe5300098a0 tx=0x7fe53002b980 comp rx=0 tx=0).stop 2024-03-28T01:05:59.633 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.631+0000 7fe549200640 1 -- 172.21.15.194:0/2528369921 shutdown_connections 2024-03-28T01:05:59.633 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.631+0000 7fe549200640 1 --2- 172.21.15.194:0/2528369921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5440b6a00 0x7fe544147020 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.634 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.631+0000 7fe549200640 1 -- 172.21.15.194:0/2528369921 >> 172.21.15.194:0/2528369921 conn(0x7fe544072760 msgr2=0x7fe544072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:59.634 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.631+0000 7fe549200640 1 -- 172.21.15.194:0/2528369921 shutdown_connections 2024-03-28T01:05:59.634 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.631+0000 7fe549200640 1 -- 172.21.15.194:0/2528369921 wait complete. 2024-03-28T01:05:59.634 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.632+0000 7fe549200640 1 Processor -- start 2024-03-28T01:05:59.634 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.632+0000 7fe549200640 1 -- start start 2024-03-28T01:05:59.637 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.633+0000 7fe549200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5440b6a00 0x7fe5441e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.637 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.633+0000 7fe549200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5440b9530 0x7fe544138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.637 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.633+0000 7fe549200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe544140cb0 0x7fe54413a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.637 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.633+0000 7fe549200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5441529b0 con 0x7fe5440b6a00 2024-03-28T01:05:59.637 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.633+0000 7fe549200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544147560 con 0x7fe5440b9530 2024-03-28T01:05:59.637 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.633+0000 7fe549200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe544152b30 con 0x7fe544140cb0 2024-03-28T01:05:59.637 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.633+0000 7fe542a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5440b9530 0x7fe544138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe543e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe544140cb0 0x7fe54413a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5440b9530 0x7fe544138820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49312/0 (socket says 172.21.15.194:49312) 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 -- 172.21.15.194:0/3213061710 learned_addr learned my addr 172.21.15.194:0/3213061710 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe543400640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5440b6a00 0x7fe5441e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 -- 172.21.15.194:0/3213061710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe544140cb0 msgr2=0x7fe54413a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe544140cb0 0x7fe54413a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 -- 172.21.15.194:0/3213061710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5440b6a00 msgr2=0x7fe5441e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5440b6a00 0x7fe5441e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 -- 172.21.15.194:0/3213061710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54413b670 con 0x7fe5440b9530 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe543400640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5440b6a00 0x7fe5441e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.634+0000 7fe542a00640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5440b9530 0x7fe544138820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe530000c00 tx=0x7fe530004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.635+0000 7fe53b400640 1 -- 172.21.15.194:0/3213061710 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530048070 con 0x7fe5440b9530 2024-03-28T01:05:59.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.635+0000 7fe53b400640 1 -- 172.21.15.194:0/3213061710 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530030640 con 0x7fe5440b9530 2024-03-28T01:05:59.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.635+0000 7fe53b400640 1 -- 172.21.15.194:0/3213061710 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530034040 con 0x7fe5440b9530 2024-03-28T01:05:59.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.635+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe544139890 con 0x7fe5440b9530 2024-03-28T01:05:59.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.635+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5441e0290 con 0x7fe5440b9530 2024-03-28T01:05:59.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.635+0000 7fe53b400640 1 -- 172.21.15.194:0/3213061710 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe530037070 con 0x7fe5440b9530 2024-03-28T01:05:59.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.635+0000 7fe53b400640 1 -- 172.21.15.194:0/3213061710 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe53004cda0 con 0x7fe5440b9530 2024-03-28T01:05:59.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.636+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe544075310 con 0x7fe5440b9530 2024-03-28T01:05:59.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.638+0000 7fe53b400640 1 -- 172.21.15.194:0/3213061710 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe530030810 con 0x7fe5440b9530 2024-03-28T01:05:59.719 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.716+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd new", "uuid": "68f5578e-d4a0-48fb-8719-b6436beb25ce", "id": 3} v 0) v1 -- 0x7fe544146bc0 con 0x7fe5440b9530 2024-03-28T01:05:59.727 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.725+0000 7fe53b400640 1 -- 172.21.15.194:0/3213061710 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68f5578e-d4a0-48fb-8719-b6436beb25ce", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe53004c3f0 con 0x7fe5440b9530 2024-03-28T01:05:59.728 INFO:teuthology.orchestra.run.smithi194.stdout:3 2024-03-28T01:05:59.729 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.726+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5440b9530 msgr2=0x7fe544138820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.729 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.726+0000 7fe549200640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5440b9530 0x7fe544138820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe530000c00 tx=0x7fe530004730 comp rx=0 tx=0).stop 2024-03-28T01:05:59.729 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.726+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 shutdown_connections 2024-03-28T01:05:59.729 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.727+0000 7fe549200640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe544140cb0 0x7fe54413a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.729 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.727+0000 7fe549200640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5440b9530 0x7fe544138820 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.729 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.727+0000 7fe549200640 1 --2- 172.21.15.194:0/3213061710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5440b6a00 0x7fe5441e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.729 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.727+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 >> 172.21.15.194:0/3213061710 conn(0x7fe544072760 msgr2=0x7fe544073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:59.730 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.727+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 shutdown_connections 2024-03-28T01:05:59.730 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.727+0000 7fe549200640 1 -- 172.21.15.194:0/3213061710 wait complete. 2024-03-28T01:05:59.744 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 7dc53f55-e2cd-476d-9a81-1e91cdfcae81 4 2024-03-28T01:05:59.786 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:59.783+0000 7f6b1563b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T01:05:59.789 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:59.786+0000 7f24ca22b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T01:05:59.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.921+0000 7f2a6b200640 1 Processor -- start 2024-03-28T01:05:59.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.921+0000 7f2a6b200640 1 -- start start 2024-03-28T01:05:59.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.922+0000 7f2a6b200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f2a640b9530 0x7f2a640b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.922+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a64140cb0 0x7f2a641410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.922+0000 7f2a6b200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a640b6300 con 0x7f2a640b4a30 2024-03-28T01:05:59.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.922+0000 7f2a6b200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a640b6570 con 0x7f2a640b9530 2024-03-28T01:05:59.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.922+0000 7f2a6b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640b5200 con 0x7f2a64140cb0 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a69e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f2a640b9530 0x7f2a640b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a69e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f2a640b9530 0x7f2a640b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:48552/0 (socket says 172.21.15.194:48552) 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a69e00640 1 -- 172.21.15.194:0/2212922773 learned_addr learned my addr 172.21.15.194:0/2212922773 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a6a800640 1 --2- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a64140cb0 0x7f2a641410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a69e00640 1 -- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a64140cb0 msgr2=0x7f2a641410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a69e00640 1 --2- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a64140cb0 0x7f2a641410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a69e00640 1 -- 172.21.15.194:0/2212922773 >> v1:172.21.15.33:6789/0 conn(0x7f2a640b4a30 legacy=0x7f2a64147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:05:59.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.923+0000 7f2a69e00640 1 -- 172.21.15.194:0/2212922773 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a640b4510 con 0x7f2a640b9530 2024-03-28T01:05:59.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.924+0000 7f2a69e00640 1 --2- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b9530 0x7f2a640b6a00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009860 tx=0x7f2a5802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95418e9db0f07851 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.924+0000 7f2a68a00640 1 -- 172.21.15.194:0/2212922773 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580317c0 con 0x7f2a640b9530 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.924+0000 7f2a68a00640 1 -- 172.21.15.194:0/2212922773 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a58037070 con 0x7f2a640b9530 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.924+0000 7f2a68a00640 1 -- 172.21.15.194:0/2212922773 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58031b80 con 0x7f2a640b9530 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 -- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b9530 msgr2=0x7f2a640b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 --2- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b9530 0x7f2a640b6a00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009860 tx=0x7f2a5802b860 comp rx=0 tx=0).stop 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 -- 172.21.15.194:0/2212922773 shutdown_connections 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 --2- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a64140cb0 0x7f2a641410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 --2- 172.21.15.194:0/2212922773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b9530 0x7f2a640b6a00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 -- 172.21.15.194:0/2212922773 >> 172.21.15.194:0/2212922773 conn(0x7f2a64072760 msgr2=0x7f2a64072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 -- 172.21.15.194:0/2212922773 shutdown_connections 2024-03-28T01:05:59.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.925+0000 7f2a6b200640 1 -- 172.21.15.194:0/2212922773 wait complete. 2024-03-28T01:05:59.929 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.927+0000 7f2a6b200640 1 Processor -- start 2024-03-28T01:05:59.929 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.927+0000 7f2a6b200640 1 -- start start 2024-03-28T01:05:59.931 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b4a30 0x7f2a640bc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.931 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a640b9530 0x7f2a641d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a64140cb0 0x7f2a641dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a6b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a641415f0 con 0x7f2a64140cb0 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a6b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640b4ec0 con 0x7f2a640b4a30 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a6b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a64141770 con 0x7f2a640b9530 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b4a30 0x7f2a640bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a640b9530 0x7f2a641d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b4a30 0x7f2a640bc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:48560/0 (socket says 172.21.15.194:48560) 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 -- 172.21.15.194:0/3379565537 learned_addr learned my addr 172.21.15.194:0/3379565537 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a6a800640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a64140cb0 0x7f2a641dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 -- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a640b9530 msgr2=0x7f2a641d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a640b9530 0x7f2a641d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.932 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 -- 172.21.15.194:0/3379565537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a64140cb0 msgr2=0x7f2a641dcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a64140cb0 0x7f2a641dcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69e00640 1 -- 172.21.15.194:0/3379565537 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a641d2a80 con 0x7f2a640b4a30 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.928+0000 7f2a69400640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a640b9530 0x7f2a641d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a69e00640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b4a30 0x7f2a640bc000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a58000c00 tx=0x7f2a580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a5e000640 1 -- 172.21.15.194:0/3379565537 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58032650 con 0x7f2a640b4a30 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a5e000640 1 -- 172.21.15.194:0/3379565537 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a580327f0 con 0x7f2a640b4a30 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a5e000640 1 -- 172.21.15.194:0/3379565537 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58032be0 con 0x7f2a640b4a30 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a641d0870 con 0x7f2a640b4a30 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a641d27a0 con 0x7f2a640b4a30 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a5e000640 1 -- 172.21.15.194:0/3379565537 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2a5804c060 con 0x7f2a640b4a30 2024-03-28T01:05:59.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.929+0000 7f2a5e000640 1 -- 172.21.15.194:0/3379565537 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2a580459a0 con 0x7f2a640b4a30 2024-03-28T01:05:59.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.930+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a64075310 con 0x7f2a640b4a30 2024-03-28T01:05:59.934 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:59.930+0000 7f6b1563b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T01:05:59.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:05:59.934+0000 7f2a5e000640 1 -- 172.21.15.194:0/3379565537 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2a58040070 con 0x7f2a640b4a30 2024-03-28T01:05:59.958 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:05:59.956+0000 7f24ca22b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T01:05:59.990 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:05:59.987+0000 7f6b1563b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T01:06:00.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.013+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7dc53f55-e2cd-476d-9a81-1e91cdfcae81", "id": 4} v 0) v1 -- 0x7f2a64145310 con 0x7f2a640b4a30 2024-03-28T01:06:00.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.020+0000 7f2a5e000640 1 -- 172.21.15.194:0/3379565537 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7dc53f55-e2cd-476d-9a81-1e91cdfcae81", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2a58045c80 con 0x7f2a640b4a30 2024-03-28T01:06:00.023 INFO:teuthology.orchestra.run.smithi194.stdout:4 2024-03-28T01:06:00.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b4a30 msgr2=0x7f2a640bc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b4a30 0x7f2a640bc000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a58000c00 tx=0x7f2a580046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 shutdown_connections 2024-03-28T01:06:00.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a64140cb0 0x7f2a641dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a640b9530 0x7f2a641d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 --2- 172.21.15.194:0/3379565537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a640b4a30 0x7f2a640bc000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 >> 172.21.15.194:0/3379565537 conn(0x7f2a64072760 msgr2=0x7f2a640bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:00.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 shutdown_connections 2024-03-28T01:06:00.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.022+0000 7f2a6b200640 1 -- 172.21.15.194:0/3379565537 wait complete. 2024-03-28T01:06:00.041 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new d1e62fa6-2e50-4354-a6d6-089469a34ae5 5 2024-03-28T01:06:00.089 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:00.087+0000 7f24ca22b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T01:06:00.125 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:00.122+0000 7f6b1563b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T01:06:00.146 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:00.144+0000 7f24ca22b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T01:06:00.245 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:00.241+0000 7f6b1563b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T01:06:00.272 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.270+0000 7f5e4dc00640 1 Processor -- start 2024-03-28T01:06:00.272 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.270+0000 7f5e4dc00640 1 -- start start 2024-03-28T01:06:00.274 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.271+0000 7f5e4dc00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f5e48140cb0 0x7f5e48141100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.274 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.271+0000 7f5e4dc00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480b6300 con 0x7f5e480b6a00 2024-03-28T01:06:00.274 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.271+0000 7f5e4dc00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480b5200 con 0x7f5e480b9530 2024-03-28T01:06:00.274 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.271+0000 7f5e4dc00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e480b6570 con 0x7f5e48140cb0 2024-03-28T01:06:00.274 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.272+0000 7f5e4d200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f5e48140cb0 0x7f5e48141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.274 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.272+0000 7f5e4d200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f5e48140cb0 0x7f5e48141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:48568/0 (socket says 172.21.15.194:48568) 2024-03-28T01:06:00.274 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.272+0000 7f5e4d200640 1 -- 172.21.15.194:0/2158340554 learned_addr learned my addr 172.21.15.194:0/2158340554 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:06:00.275 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.272+0000 7f5e4d200640 1 -- 172.21.15.194:0/2158340554 >> v1:172.21.15.194:6789/0 conn(0x7f5e480b9530 legacy=0x7f5e480b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.275 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.272+0000 7f5e4d200640 1 -- 172.21.15.194:0/2158340554 >> v1:172.21.15.33:6789/0 conn(0x7f5e480b6a00 legacy=0x7f5e48147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:00.275 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.272+0000 7f5e4d200640 1 -- 172.21.15.194:0/2158340554 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480b4510 con 0x7f5e48140cb0 2024-03-28T01:06:00.275 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.273+0000 7f5e4d200640 1 --2- 172.21.15.194:0/2158340554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e48140cb0 0x7f5e48141100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009e50 tx=0x7f5e4002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7537bc09695bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:00.276 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.273+0000 7f5e46a00640 1 -- 172.21.15.194:0/2158340554 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4002ca80 con 0x7f5e48140cb0 2024-03-28T01:06:00.276 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.273+0000 7f5e46a00640 1 -- 172.21.15.194:0/2158340554 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4002cc20 con 0x7f5e48140cb0 2024-03-28T01:06:00.276 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.273+0000 7f5e46a00640 1 -- 172.21.15.194:0/2158340554 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e400343f0 con 0x7f5e48140cb0 2024-03-28T01:06:00.276 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.274+0000 7f5e4dc00640 1 -- 172.21.15.194:0/2158340554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e48140cb0 msgr2=0x7f5e48141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.276 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.274+0000 7f5e4dc00640 1 --2- 172.21.15.194:0/2158340554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e48140cb0 0x7f5e48141100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009e50 tx=0x7f5e4002bee0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.277 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.274+0000 7f5e4dc00640 1 -- 172.21.15.194:0/2158340554 shutdown_connections 2024-03-28T01:06:00.277 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.274+0000 7f5e4dc00640 1 --2- 172.21.15.194:0/2158340554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e48140cb0 0x7f5e48141100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.277 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.274+0000 7f5e4dc00640 1 -- 172.21.15.194:0/2158340554 >> 172.21.15.194:0/2158340554 conn(0x7f5e48072760 msgr2=0x7f5e48072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:00.277 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.274+0000 7f5e4dc00640 1 -- 172.21.15.194:0/2158340554 shutdown_connections 2024-03-28T01:06:00.277 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.274+0000 7f5e4dc00640 1 -- 172.21.15.194:0/2158340554 wait complete. 2024-03-28T01:06:00.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 Processor -- start 2024-03-28T01:06:00.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 -- start start 2024-03-28T01:06:00.279 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480b6a00 0x7f5e4813d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.279 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e480b9530 0x7f5e481dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.279 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e48140cb0 0x7f5e481dd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e481529b0 con 0x7f5e480b6a00 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48141640 con 0x7f5e480b9530 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.276+0000 7f5e4dc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48152b30 con 0x7f5e48140cb0 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e47400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e480b9530 0x7f5e481dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e47400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e480b9530 0x7f5e481dcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:48572/0 (socket says 172.21.15.194:48572) 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e4d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e48140cb0 0x7f5e481dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e47400640 1 -- 172.21.15.194:0/859246794 learned_addr learned my addr 172.21.15.194:0/859246794 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e47e00640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480b6a00 0x7f5e4813d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e4d200640 1 -- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e480b9530 msgr2=0x7f5e481dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e4d200640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e480b9530 0x7f5e481dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e4d200640 1 -- 172.21.15.194:0/859246794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480b6a00 msgr2=0x7f5e4813d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e4d200640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480b6a00 0x7f5e4813d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.280 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e4d200640 1 -- 172.21.15.194:0/859246794 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4813edc0 con 0x7f5e48140cb0 2024-03-28T01:06:00.281 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e47400640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e480b9530 0x7f5e481dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:00.281 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.277+0000 7f5e4d200640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e48140cb0 0x7f5e481dd480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e40032000 tx=0x7f5e40004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:00.282 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.278+0000 7f5e3fe00640 1 -- 172.21.15.194:0/859246794 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40043020 con 0x7f5e48140cb0 2024-03-28T01:06:00.282 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.278+0000 7f5e3fe00640 1 -- 172.21.15.194:0/859246794 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4003e450 con 0x7f5e48140cb0 2024-03-28T01:06:00.282 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.278+0000 7f5e3fe00640 1 -- 172.21.15.194:0/859246794 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40009510 con 0x7f5e48140cb0 2024-03-28T01:06:00.282 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.278+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e48144bc0 con 0x7f5e48140cb0 2024-03-28T01:06:00.282 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.278+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4813f030 con 0x7f5e48140cb0 2024-03-28T01:06:00.282 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.278+0000 7f5e3fe00640 1 -- 172.21.15.194:0/859246794 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5e40046020 con 0x7f5e48140cb0 2024-03-28T01:06:00.282 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.278+0000 7f5e3fe00640 1 -- 172.21.15.194:0/859246794 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f5e4002ca90 con 0x7f5e48140cb0 2024-03-28T01:06:00.282 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:00.279+0000 7f24ca22b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T01:06:00.283 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.279+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e48075310 con 0x7f5e48140cb0 2024-03-28T01:06:00.283 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.281+0000 7f5e3fe00640 1 -- 172.21.15.194:0/859246794 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5e4004d030 con 0x7f5e48140cb0 2024-03-28T01:06:00.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.359+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d1e62fa6-2e50-4354-a6d6-089469a34ae5", "id": 5} v 0) v1 -- 0x7f5e480baea0 con 0x7f5e48140cb0 2024-03-28T01:06:00.369 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.366+0000 7f5e3fe00640 1 -- 172.21.15.194:0/859246794 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1e62fa6-2e50-4354-a6d6-089469a34ae5", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5e4003e820 con 0x7f5e48140cb0 2024-03-28T01:06:00.369 INFO:teuthology.orchestra.run.smithi194.stdout:5 2024-03-28T01:06:00.370 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e48140cb0 msgr2=0x7f5e481dd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.370 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e48140cb0 0x7f5e481dd480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e40032000 tx=0x7f5e40004a40 comp rx=0 tx=0).stop 2024-03-28T01:06:00.370 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 shutdown_connections 2024-03-28T01:06:00.370 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e48140cb0 0x7f5e481dd480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.371 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e480b9530 0x7f5e481dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.371 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 --2- 172.21.15.194:0/859246794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480b6a00 0x7f5e4813d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.371 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 >> 172.21.15.194:0/859246794 conn(0x7f5e48072760 msgr2=0x7f5e480bb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:00.371 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 shutdown_connections 2024-03-28T01:06:00.371 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.368+0000 7f5e4dc00640 1 -- 172.21.15.194:0/859246794 wait complete. 2024-03-28T01:06:00.394 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new e965ff9e-93f4-41ae-be0f-040bf76ae514 6 2024-03-28T01:06:00.404 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:00.402+0000 7f24ca22b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T01:06:00.432 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:00.428+0000 7f6b1563b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T01:06:00.484 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:00.480+0000 7f6b1563b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T01:06:00.591 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:00.588+0000 7f24ca22b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T01:06:00.596 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.594+0000 7f740f200640 1 Processor -- start 2024-03-28T01:06:00.596 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.594+0000 7f740f200640 1 -- start start 2024-03-28T01:06:00.598 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.595+0000 7f740f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7410071d60 0x7f7410072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.598 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.595+0000 7f740f200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7410141040 0x7f74101414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.598 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.595+0000 7f740f200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f741006ba70 0x7f741006bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.598 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.595+0000 7f740f200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741006c410 con 0x7f741006ba70 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.595+0000 7f740f200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741006c610 con 0x7f7410141040 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.595+0000 7f740f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74100729d0 con 0x7f7410071d60 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7410071d60 0x7f7410072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740d400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7410141040 0x7f74101414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740d400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7410141040 0x7f74101414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:48584/0 (socket says 172.21.15.194:48584) 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7410071d60 0x7f7410072140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:51096/0 (socket says 172.21.15.194:51096) 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740d400640 1 -- 172.21.15.194:0/3688207497 learned_addr learned my addr 172.21.15.194:0/3688207497 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740e800640 1 --2- 172.21.15.194:0/3688207497 >> v2:172.21.15.33:3300/0 conn(0x7f741006ba70 0x7f741006bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740de00640 1 -- 172.21.15.194:0/3688207497 >> v2:172.21.15.194:3300/0 conn(0x7f7410141040 msgr2=0x7f74101414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740de00640 1 --2- 172.21.15.194:0/3688207497 >> v2:172.21.15.194:3300/0 conn(0x7f7410141040 0x7f74101414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740de00640 1 -- 172.21.15.194:0/3688207497 >> v2:172.21.15.33:3300/0 conn(0x7f741006ba70 msgr2=0x7f741006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740de00640 1 --2- 172.21.15.194:0/3688207497 >> v2:172.21.15.33:3300/0 conn(0x7f741006ba70 0x7f741006bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740de00640 1 -- 172.21.15.194:0/3688207497 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7410072cb0 con 0x7f7410071d60 2024-03-28T01:06:00.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.596+0000 7f740d400640 1 --2- 172.21.15.194:0/3688207497 >> v2:172.21.15.194:3300/0 conn(0x7f7410141040 0x7f74101414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.597+0000 7f740de00640 1 --2- 172.21.15.194:0/3688207497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7410071d60 0x7f7410072140 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74100b5b60 tx=0x7f740002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c6510090c80b40 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.597+0000 7f740ca00640 1 -- 172.21.15.194:0/3688207497 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400037070 con 0x7f7410071d60 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.597+0000 7f740ca00640 1 -- 172.21.15.194:0/3688207497 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7400030710 con 0x7f7410071d60 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.597+0000 7f740ca00640 1 -- 172.21.15.194:0/3688207497 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400030ae0 con 0x7f7410071d60 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 -- 172.21.15.194:0/3688207497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7410071d60 msgr2=0x7f7410072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 --2- 172.21.15.194:0/3688207497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7410071d60 0x7f7410072140 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74100b5b60 tx=0x7f740002b710 comp rx=0 tx=0).stop 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 -- 172.21.15.194:0/3688207497 shutdown_connections 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 --2- 172.21.15.194:0/3688207497 >> v2:172.21.15.33:3300/0 conn(0x7f741006ba70 0x7f741006bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 --2- 172.21.15.194:0/3688207497 >> v2:172.21.15.194:3300/0 conn(0x7f7410141040 0x7f74101414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 --2- 172.21.15.194:0/3688207497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7410071d60 0x7f7410072140 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 -- 172.21.15.194:0/3688207497 >> 172.21.15.194:0/3688207497 conn(0x7f74100b9860 msgr2=0x7f74100b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:00.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 -- 172.21.15.194:0/3688207497 shutdown_connections 2024-03-28T01:06:00.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.598+0000 7f740f200640 1 -- 172.21.15.194:0/3688207497 wait complete. 2024-03-28T01:06:00.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.600+0000 7f740f200640 1 Processor -- start 2024-03-28T01:06:00.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.600+0000 7f740f200640 1 -- start start 2024-03-28T01:06:00.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.600+0000 7f740f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f741006ba70 0x7f74101e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.600+0000 7f740f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7410071d60 0x7f741013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7410141040 0x7f741013d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741006d2e0 con 0x7f7410141040 2024-03-28T01:06:00.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741006d160 con 0x7f7410071d60 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741014a710 con 0x7f741006ba70 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f741006ba70 0x7f74101e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f741006ba70 0x7f74101e0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:51108/0 (socket says 172.21.15.194:51108) 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 -- 172.21.15.194:0/1104552921 learned_addr learned my addr 172.21.15.194:0/1104552921 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740d400640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7410071d60 0x7f741013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740e800640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7410141040 0x7f741013d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 -- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7410071d60 msgr2=0x7f741013ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7410071d60 0x7f741013ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 -- 172.21.15.194:0/1104552921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7410141040 msgr2=0x7f741013d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7410141040 0x7f741013d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.601+0000 7f740de00640 1 -- 172.21.15.194:0/1104552921 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7410142080 con 0x7f741006ba70 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f740de00640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f741006ba70 0x7f74101e0680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7400000c00 tx=0x7f74000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:00.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f7406000640 1 -- 172.21.15.194:0/1104552921 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400041020 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f7406000640 1 -- 172.21.15.194:0/1104552921 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740003c890 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7410140810 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f7406000640 1 -- 172.21.15.194:0/1104552921 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400037040 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74101dfe80 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f7406000640 1 -- 172.21.15.194:0/1104552921 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7400044020 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.602+0000 7f7406000640 1 -- 172.21.15.194:0/1104552921 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f740003dad0 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.603+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7410075310 con 0x7f741006ba70 2024-03-28T01:06:00.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.606+0000 7f7406000640 1 -- 172.21.15.194:0/1104552921 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f740004b030 con 0x7f741006ba70 2024-03-28T01:06:00.645 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:00.642+0000 7f24ca22b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T01:06:00.667 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:00.664+0000 7f6b1563b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T01:06:00.687 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.685+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e965ff9e-93f4-41ae-be0f-040bf76ae514", "id": 6} v 0) v1 -- 0x7f7410067630 con 0x7f741006ba70 2024-03-28T01:06:00.696 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.693+0000 7f7406000640 1 -- 172.21.15.194:0/1104552921 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e965ff9e-93f4-41ae-be0f-040bf76ae514", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f740003dd20 con 0x7f741006ba70 2024-03-28T01:06:00.696 INFO:teuthology.orchestra.run.smithi194.stdout:6 2024-03-28T01:06:00.697 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f741006ba70 msgr2=0x7f74101e0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.697 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f741006ba70 0x7f74101e0680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7400000c00 tx=0x7f74000046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.697 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 shutdown_connections 2024-03-28T01:06:00.697 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7410141040 0x7f741013d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.697 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7410071d60 0x7f741013ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.698 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 --2- 172.21.15.194:0/1104552921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f741006ba70 0x7f74101e0680 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.698 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 >> 172.21.15.194:0/1104552921 conn(0x7f74100b9860 msgr2=0x7f7410073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:00.698 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 shutdown_connections 2024-03-28T01:06:00.698 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.695+0000 7f740f200640 1 -- 172.21.15.194:0/1104552921 wait complete. 2024-03-28T01:06:00.713 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new d9c5796e-d4cd-44d4-80ea-3df37f708427 7 2024-03-28T01:06:00.829 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:00.827+0000 7f24ca22b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T01:06:00.895 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.893+0000 7f7d68e00640 1 Processor -- start 2024-03-28T01:06:00.896 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.893+0000 7f7d68e00640 1 -- start start 2024-03-28T01:06:00.897 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.894+0000 7f7d68e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7d640b9530 0x7f7d640b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.897 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.894+0000 7f7d68e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d640b4a30 0x7f7d64147020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.894+0000 7f7d68e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d64140cb0 0x7f7d641410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.894+0000 7f7d68e00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d640b6570 con 0x7f7d64140cb0 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.894+0000 7f7d68e00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d640b6300 con 0x7f7d640b9530 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.894+0000 7f7d68e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640b5200 con 0x7f7d640b4a30 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d5aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d640b4a30 0x7f7d64147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d63400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7d640b9530 0x7f7d640b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d5aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d640b4a30 0x7f7d64147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:51112/0 (socket says 172.21.15.194:51112) 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d5aa00640 1 -- 172.21.15.194:0/666475219 learned_addr learned my addr 172.21.15.194:0/666475219 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d63e00640 1 --2- 172.21.15.194:0/666475219 >> v2:172.21.15.33:3300/0 conn(0x7f7d64140cb0 0x7f7d641410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d63400640 1 -- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d640b4a30 msgr2=0x7f7d64147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d63400640 1 --2- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d640b4a30 0x7f7d64147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d63400640 1 -- 172.21.15.194:0/666475219 >> v2:172.21.15.33:3300/0 conn(0x7f7d64140cb0 msgr2=0x7f7d641410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:00.899 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d63400640 1 --2- 172.21.15.194:0/666475219 >> v2:172.21.15.33:3300/0 conn(0x7f7d64140cb0 0x7f7d641410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.899 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d63400640 1 -- 172.21.15.194:0/666475219 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640b4510 con 0x7f7d640b9530 2024-03-28T01:06:00.899 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.895+0000 7f7d5aa00640 1 --2- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d640b4a30 0x7f7d64147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.896+0000 7f7d63400640 1 --2- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d640b6a00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d640b9d20 tx=0x7f7d5402bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5144b145a6151a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.897+0000 7f7d62a00640 1 -- 172.21.15.194:0/666475219 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54039070 con 0x7f7d640b9530 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.897+0000 7f7d62a00640 1 -- 172.21.15.194:0/666475219 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5402cb90 con 0x7f7d640b9530 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.897+0000 7f7d62a00640 1 -- 172.21.15.194:0/666475219 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54034440 con 0x7f7d640b9530 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.897+0000 7f7d68e00640 1 -- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 msgr2=0x7f7d640b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.897+0000 7f7d68e00640 1 --2- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d640b6a00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d640b9d20 tx=0x7f7d5402bde0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.898+0000 7f7d68e00640 1 -- 172.21.15.194:0/666475219 shutdown_connections 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.898+0000 7f7d68e00640 1 --2- 172.21.15.194:0/666475219 >> v2:172.21.15.33:3300/0 conn(0x7f7d64140cb0 0x7f7d641410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.898+0000 7f7d68e00640 1 --2- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d640b4a30 0x7f7d64147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.900 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.898+0000 7f7d68e00640 1 --2- 172.21.15.194:0/666475219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d640b6a00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.898+0000 7f7d68e00640 1 -- 172.21.15.194:0/666475219 >> 172.21.15.194:0/666475219 conn(0x7f7d64072760 msgr2=0x7f7d64072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:00.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.898+0000 7f7d68e00640 1 -- 172.21.15.194:0/666475219 shutdown_connections 2024-03-28T01:06:00.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.898+0000 7f7d68e00640 1 -- 172.21.15.194:0/666475219 wait complete. 2024-03-28T01:06:00.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.899+0000 7f7d68e00640 1 Processor -- start 2024-03-28T01:06:00.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.899+0000 7f7d68e00640 1 -- start start 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d68e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d640b4a30 0x7f7d641e0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d68e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d641d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d68e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d64140cb0 0x7f7d6413cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d68e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64147560 con 0x7f7d640b4a30 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d68e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640b4e90 con 0x7f7d640b9530 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d68e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64141620 con 0x7f7d64140cb0 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d5aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d641d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d64140cb0 0x7f7d6413cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d64140cb0 0x7f7d6413cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:51124/0 (socket says 172.21.15.194:51124) 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d5aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d641d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:48608/0 (socket says 172.21.15.194:48608) 2024-03-28T01:06:00.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d63e00640 1 -- 172.21.15.194:0/2311314505 learned_addr learned my addr 172.21.15.194:0/2311314505 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.900+0000 7f7d63400640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d640b4a30 0x7f7d641e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5aa00640 1 -- 172.21.15.194:0/2311314505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d64140cb0 msgr2=0x7f7d6413cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5aa00640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d64140cb0 0x7f7d6413cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5aa00640 1 -- 172.21.15.194:0/2311314505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d640b4a30 msgr2=0x7f7d641e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5aa00640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d640b4a30 0x7f7d641e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5aa00640 1 -- 172.21.15.194:0/2311314505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d641d2e80 con 0x7f7d640b9530 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d63400640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d640b4a30 0x7f7d641e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5aa00640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d641d0990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e900 tx=0x7f7d4c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5be00640 1 -- 172.21.15.194:0/2311314505 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00ca00 con 0x7f7d640b9530 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5be00640 1 -- 172.21.15.194:0/2311314505 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c00cba0 con 0x7f7d640b9530 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d5be00640 1 -- 172.21.15.194:0/2311314505 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004030 con 0x7f7d640b9530 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d641cfc50 con 0x7f7d640b9530 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.901+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d641ceeb0 con 0x7f7d640b9530 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.902+0000 7f7d5be00640 1 -- 172.21.15.194:0/2311314505 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7d4c005990 con 0x7f7d640b9530 2024-03-28T01:06:00.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.902+0000 7f7d5be00640 1 -- 172.21.15.194:0/2311314505 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f7d4c0093d0 con 0x7f7d640b9530 2024-03-28T01:06:00.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.902+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d64075310 con 0x7f7d640b9530 2024-03-28T01:06:00.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.906+0000 7f7d5be00640 1 -- 172.21.15.194:0/2311314505 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7d4c017020 con 0x7f7d640b9530 2024-03-28T01:06:00.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.985+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9c5796e-d4cd-44d4-80ea-3df37f708427", "id": 7} v 0) v1 -- 0x7f7d640b5fb0 con 0x7f7d640b9530 2024-03-28T01:06:00.994 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.992+0000 7f7d5be00640 1 -- 172.21.15.194:0/2311314505 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9c5796e-d4cd-44d4-80ea-3df37f708427", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d4c005c80 con 0x7f7d640b9530 2024-03-28T01:06:00.995 INFO:teuthology.orchestra.run.smithi194.stdout:7 2024-03-28T01:06:00.996 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.993+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 msgr2=0x7f7d641d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:00.996 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.993+0000 7f7d68e00640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d641d0990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e900 tx=0x7f7d4c00edc0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.996 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.994+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 shutdown_connections 2024-03-28T01:06:00.996 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.994+0000 7f7d68e00640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d64140cb0 0x7f7d6413cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.996 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.994+0000 7f7d68e00640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d640b9530 0x7f7d641d0990 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.996 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.994+0000 7f7d68e00640 1 --2- 172.21.15.194:0/2311314505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d640b4a30 0x7f7d641e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:00.997 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.994+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 >> 172.21.15.194:0/2311314505 conn(0x7f7d64072760 msgr2=0x7f7d640bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:00.997 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.994+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 shutdown_connections 2024-03-28T01:06:00.997 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-28T01:06:00.994+0000 7f7d68e00640 1 -- 172.21.15.194:0/2311314505 wait complete. 2024-03-28T01:06:01.012 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-28T01:06:01.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-28T01:06:01.020 INFO:tasks.ceph.osd.0:Started 2024-03-28T01:06:01.020 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-28T01:06:01.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-28T01:06:01.027 INFO:tasks.ceph.osd.1:Started 2024-03-28T01:06:01.027 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-28T01:06:01.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-28T01:06:01.031 INFO:tasks.ceph.osd.2:Started 2024-03-28T01:06:01.031 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-28T01:06:01.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-28T01:06:01.036 INFO:tasks.ceph.osd.3:Started 2024-03-28T01:06:01.037 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-28T01:06:01.037 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-28T01:06:01.041 INFO:tasks.ceph.osd.4:Started 2024-03-28T01:06:01.041 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-28T01:06:01.041 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-28T01:06:01.045 INFO:tasks.ceph.osd.5:Started 2024-03-28T01:06:01.045 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-28T01:06:01.045 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-28T01:06:01.049 INFO:tasks.ceph.osd.6:Started 2024-03-28T01:06:01.050 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-28T01:06:01.050 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-28T01:06:01.056 INFO:tasks.ceph.osd.7:Started 2024-03-28T01:06:01.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.268+0000 7f696b800640 1 Processor -- start 2024-03-28T01:06:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.268+0000 7f696b800640 1 -- start start 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f696b800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69640b3be0 0x7f69640ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f696b800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f69640b25e0 0x7f69640b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f696b800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696406d7c0 con 0x7f696406d3e0 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f696b800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69640aed20 con 0x7f69640b25e0 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f696b800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69640b4a80 con 0x7f69640b3be0 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f6968e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f696406d3e0 0x7f69640b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41144/0 (socket says 172.21.15.33:41144) 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f6968e00640 1 -- 172.21.15.33:0/2899760225 learned_addr learned my addr 172.21.15.33:0/2899760225 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.269+0000 7f6969800640 1 --2- 172.21.15.33:0/2899760225 >> v2:172.21.15.194:3300/0 conn(0x7f69640b25e0 0x7f69640b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6963e00640 1 --2- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69640b3be0 0x7f69640ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:01.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6963400640 1 -- 172.21.15.33:0/2899760225 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304018390 0 0) 0x7f696406d7c0 con 0x7f696406d3e0 2024-03-28T01:06:01.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6963400640 1 -- 172.21.15.33:0/2899760225 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f694c003520 con 0x7f696406d3e0 2024-03-28T01:06:01.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6969800640 1 -- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69640b3be0 msgr2=0x7f69640ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:01.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6969800640 1 --2- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69640b3be0 0x7f69640ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6969800640 1 -- 172.21.15.33:0/2899760225 >> v1:172.21.15.33:6789/0 conn(0x7f696406d3e0 legacy=0x7f69640b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6969800640 1 -- 172.21.15.33:0/2899760225 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69640afc70 con 0x7f69640b25e0 2024-03-28T01:06:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.270+0000 7f6963e00640 1 --2- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69640b3be0 0x7f69640ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.272+0000 7f6969800640 1 --2- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69640b25e0 0x7f69640b2a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f695c009d90 tx=0x7f695c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeade4d4fa5fe4f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.272+0000 7f6963400640 1 -- 172.21.15.33:0/2899760225 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c039070 con 0x7f69640b25e0 2024-03-28T01:06:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.272+0000 7f6963400640 1 -- 172.21.15.33:0/2899760225 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c02cbc0 con 0x7f69640b25e0 2024-03-28T01:06:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.272+0000 7f6963400640 1 -- 172.21.15.33:0/2899760225 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c034440 con 0x7f69640b25e0 2024-03-28T01:06:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 -- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69640b25e0 msgr2=0x7f69640b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 --2- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69640b25e0 0x7f69640b2a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f695c009d90 tx=0x7f695c02be10 comp rx=0 tx=0).stop 2024-03-28T01:06:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 -- 172.21.15.33:0/2899760225 shutdown_connections 2024-03-28T01:06:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 --2- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69640b25e0 0x7f69640b2a40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 --2- 172.21.15.33:0/2899760225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69640b3be0 0x7f69640ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 -- 172.21.15.33:0/2899760225 >> 172.21.15.33:0/2899760225 conn(0x7f696406b030 msgr2=0x7f696406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 -- 172.21.15.33:0/2899760225 shutdown_connections 2024-03-28T01:06:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.273+0000 7f696b800640 1 -- 172.21.15.33:0/2899760225 wait complete. 2024-03-28T01:06:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.274+0000 7f696b800640 1 Processor -- start 2024-03-28T01:06:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.274+0000 7f696b800640 1 -- start start 2024-03-28T01:06:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f696b800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f696406d3e0 0x7f69641dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f696b800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69640b25e0 0x7f69640710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f696b800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69641445a0 0x7f6964144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f696b800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69640b2f80 con 0x7f69640b25e0 2024-03-28T01:06:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f696b800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69640b4fb0 con 0x7f69641445a0 2024-03-28T01:06:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f696b800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6964072f00 con 0x7f696406d3e0 2024-03-28T01:06:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f6963e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69640b25e0 0x7f69640710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f6963e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69640b25e0 0x7f69640710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55424/0 (socket says 172.21.15.33:55424) 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.275+0000 7f6963e00640 1 -- 172.21.15.33:0/1739955230 learned_addr learned my addr 172.21.15.33:0/1739955230 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6969800640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69641445a0 0x7f6964144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6968e00640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f696406d3e0 0x7f69641dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6963e00640 1 -- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f696406d3e0 msgr2=0x7f69641dfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6963e00640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f696406d3e0 0x7f69641dfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6963e00640 1 -- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69641445a0 msgr2=0x7f6964144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6963e00640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69641445a0 0x7f6964144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6963e00640 1 -- 172.21.15.33:0/1739955230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69641e0320 con 0x7f69640b25e0 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6968e00640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f696406d3e0 0x7f69641dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6963e00640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69640b25e0 0x7f69640710f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f695000e990 tx=0x7f695000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6960c00640 1 -- 172.21.15.33:0/1739955230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950017070 con 0x7f69640b25e0 2024-03-28T01:06:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6960c00640 1 -- 172.21.15.33:0/1739955230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695000c980 con 0x7f69640b25e0 2024-03-28T01:06:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69641e0600 con 0x7f69640b25e0 2024-03-28T01:06:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f6960c00640 1 -- 172.21.15.33:0/1739955230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000cd50 con 0x7f69640b25e0 2024-03-28T01:06:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.276+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6964071d80 con 0x7f69640b25e0 2024-03-28T01:06:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.277+0000 7f6960c00640 1 -- 172.21.15.33:0/1739955230 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f695001a070 con 0x7f69640b25e0 2024-03-28T01:06:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.277+0000 7f6960c00640 1 -- 172.21.15.33:0/1739955230 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f695001cb60 con 0x7f69640b25e0 2024-03-28T01:06:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.278+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6964075380 con 0x7f69640b25e0 2024-03-28T01:06:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.281+0000 7f6960c00640 1 -- 172.21.15.33:0/1739955230 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f695001c450 con 0x7f69640b25e0 2024-03-28T01:06:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.347+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6964146a70 con 0x7f69640b25e0 2024-03-28T01:06:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.347+0000 7f6960c00640 1 -- 172.21.15.33:0/1739955230 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f695001c630 con 0x7f69640b25e0 2024-03-28T01:06:01.351 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:01.351 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:00.987503+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:01.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.348+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69640b25e0 msgr2=0x7f69640710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:01.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.348+0000 7f696b800640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69640b25e0 0x7f69640710f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f695000e990 tx=0x7f695000ee50 comp rx=0 tx=0).stop 2024-03-28T01:06:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.348+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 shutdown_connections 2024-03-28T01:06:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.348+0000 7f696b800640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69641445a0 0x7f6964144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.348+0000 7f696b800640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69640b25e0 0x7f69640710f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.348+0000 7f696b800640 1 --2- 172.21.15.33:0/1739955230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f696406d3e0 0x7f69641dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.348+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 >> 172.21.15.33:0/1739955230 conn(0x7f696406b030 msgr2=0x7f696406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.349+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 shutdown_connections 2024-03-28T01:06:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:01.349+0000 7f696b800640 1 -- 172.21.15.33:0/1739955230 wait complete. 2024-03-28T01:06:01.367 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-28T01:06:01.367 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-28T01:06:01.723 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-03-28T01:06:01.721+0000 7f99eb138640 -1 Falling back to public interface 2024-03-28T01:06:01.724 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-03-28T01:06:01.721+0000 7f057219a640 -1 Falling back to public interface 2024-03-28T01:06:01.724 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-03-28T01:06:01.722+0000 7f0570c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-03-28T01:06:01.742 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-03-28T01:06:01.738+0000 7f6d4e0a6640 -1 Falling back to public interface 2024-03-28T01:06:01.746 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-03-28T01:06:01.743+0000 7ffb5b00f640 -1 Falling back to public interface 2024-03-28T01:06:01.751 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-03-28T01:06:01.748+0000 7f0f786f9640 -1 Falling back to public interface 2024-03-28T01:06:01.753 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-03-28T01:06:01.750+0000 7f46ef01c640 -1 Falling back to public interface 2024-03-28T01:06:01.754 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-03-28T01:06:01.750+0000 7f119ae19640 -1 Falling back to public interface 2024-03-28T01:06:01.766 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-03-28T01:06:01.764+0000 7f13e8ed7640 -1 Falling back to public interface 2024-03-28T01:06:01.827 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:01.824+0000 7f6b1563b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T01:06:01.960 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:01.957+0000 7f24ca22b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T01:06:02.153 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:02.150+0000 7f6b1563b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T01:06:02.173 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:02.221 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:02.218+0000 7f6b1563b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T01:06:02.275 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:02.273+0000 7f24ca22b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T01:06:02.332 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.329+0000 7fe026800640 1 Processor -- start 2024-03-28T01:06:02.333 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.329+0000 7fe026800640 1 -- start start 2024-03-28T01:06:02.334 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.330+0000 7fe026800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe0200b5240 0x7fe0200b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:02.334 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.330+0000 7fe026800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe02014a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:02.334 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.330+0000 7fe026800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200b5620 con 0x7fe0201416c0 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.330+0000 7fe026800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200b9bc0 con 0x7fe0200b5240 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.330+0000 7fe026800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200b6760 con 0x7fe020141260 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.330+0000 7fe025400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe0200b5240 0x7fe0200b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe025400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe0200b5240 0x7fe0200b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55426/0 (socket says 172.21.15.33:55426) 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe025400640 1 -- 172.21.15.33:0/469170244 learned_addr learned my addr 172.21.15.33:0/469170244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe024a00640 1 --2- 172.21.15.33:0/469170244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe02014a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe025400640 1 -- 172.21.15.33:0/469170244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 msgr2=0x7fe02014a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe025400640 1 --2- 172.21.15.33:0/469170244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe02014a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe025400640 1 -- 172.21.15.33:0/469170244 >> v1:172.21.15.194:6789/0 conn(0x7fe0201416c0 legacy=0x7fe0200b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe025400640 1 -- 172.21.15.33:0/469170244 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200b9850 con 0x7fe0200b5240 2024-03-28T01:06:02.335 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.331+0000 7fe024a00640 1 --2- 172.21.15.33:0/469170244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe02014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:02.336 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.332+0000 7fe025400640 1 --2- 172.21.15.33:0/469170244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0200b3e00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe014009870 tx=0x7fe01402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c390dba4a8480424 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:02.336 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.332+0000 7fe01fe00640 1 -- 172.21.15.33:0/469170244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014035020 con 0x7fe0200b5240 2024-03-28T01:06:02.336 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.332+0000 7fe01fe00640 1 -- 172.21.15.33:0/469170244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe014031640 con 0x7fe0200b5240 2024-03-28T01:06:02.336 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.332+0000 7fe01fe00640 1 -- 172.21.15.33:0/469170244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01402d040 con 0x7fe0200b5240 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 -- 172.21.15.33:0/469170244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 msgr2=0x7fe0200b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 --2- 172.21.15.33:0/469170244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0200b3e00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe014009870 tx=0x7fe01402b8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 -- 172.21.15.33:0/469170244 shutdown_connections 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 --2- 172.21.15.33:0/469170244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe02014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 --2- 172.21.15.33:0/469170244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0200b3e00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 -- 172.21.15.33:0/469170244 >> 172.21.15.33:0/469170244 conn(0x7fe0200729b0 msgr2=0x7fe020072dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 -- 172.21.15.33:0/469170244 shutdown_connections 2024-03-28T01:06:02.337 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.333+0000 7fe026800640 1 -- 172.21.15.33:0/469170244 wait complete. 2024-03-28T01:06:02.338 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:02.335+0000 7f24ca22b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T01:06:02.339 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.334+0000 7fe026800640 1 Processor -- start 2024-03-28T01:06:02.339 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.335+0000 7fe026800640 1 -- start start 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.335+0000 7fe026800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0201dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe026800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe020144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe026800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0201416c0 0x7fe020143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe026800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200b6d70 con 0x7fe0200b5240 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe026800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200b6bf0 con 0x7fe0201416c0 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe026800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02014ace0 con 0x7fe020141260 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0201dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0201dd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55442/0 (socket says 172.21.15.33:55442) 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 -- 172.21.15.33:0/4245506942 learned_addr learned my addr 172.21.15.33:0/4245506942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:02.341 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe024a00640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe020144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025e00640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0201416c0 0x7fe020143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 -- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 msgr2=0x7fe020144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe020144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 -- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0201416c0 msgr2=0x7fe020143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0201416c0 0x7fe020143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025400640 1 -- 172.21.15.33:0/4245506942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0201dc870 con 0x7fe0200b5240 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.336+0000 7fe025e00640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0201416c0 0x7fe020143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe025400640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0201dd2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe01402b930 tx=0x7fe0140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014040070 con 0x7fe0200b5240 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe014039c10 con 0x7fe0200b5240 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014035040 con 0x7fe0200b5240 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe020144a50 con 0x7fe0200b5240 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0201dcdd0 con 0x7fe0200b5240 2024-03-28T01:06:02.342 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe01404b060 con 0x7fe0200b5240 2024-03-28T01:06:02.343 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.337+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe01403acd0 con 0x7fe0200b5240 2024-03-28T01:06:02.343 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.338+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe020075410 con 0x7fe0200b5240 2024-03-28T01:06:02.344 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.340+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe014063dd0 con 0x7fe0200b5240 2024-03-28T01:06:02.363 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:02.360+0000 7f6b1563b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T01:06:02.372 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.368+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 2) v1 ==== 45093+0+0 (secure 0 0 0) 0x7fe014038020 con 0x7fe0200b5240 2024-03-28T01:06:02.421 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.417+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0201dd010 con 0x7fe0200b5240 2024-03-28T01:06:02.422 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.419+0000 7fe01d600640 1 -- 172.21.15.33:0/4245506942 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fe020144a50 con 0x7fe0200b5240 2024-03-28T01:06:02.423 INFO:teuthology.misc.health.smithi033.stdout: 2024-03-28T01:06:02.423 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:00.987503+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:02.424 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.420+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 msgr2=0x7fe0201dd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:02.424 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.420+0000 7fe026800640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0201dd2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe01402b930 tx=0x7fe0140046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.424 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.420+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 shutdown_connections 2024-03-28T01:06:02.424 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.420+0000 7fe026800640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0201416c0 0x7fe020143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.425 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.420+0000 7fe026800640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe020141260 0x7fe020144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.425 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.420+0000 7fe026800640 1 --2- 172.21.15.33:0/4245506942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0200b5240 0x7fe0201dd2f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:02.425 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.420+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 >> 172.21.15.33:0/4245506942 conn(0x7fe0200729b0 msgr2=0x7fe0200ba8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:02.425 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.421+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 shutdown_connections 2024-03-28T01:06:02.425 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:02.421+0000 7fe026800640 1 -- 172.21.15.33:0/4245506942 wait complete. 2024-03-28T01:06:02.437 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-28T01:06:02.475 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-03-28T01:06:02.472+0000 7f24ca22b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T01:06:03.403 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:03.399+0000 7f6ab5e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-28T01:06:03.404 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-03-28T01:06:03.401+0000 7f6ab5e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-28T01:06:04.822 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-03-28T01:06:04.818+0000 7f119ae19640 -1 osd.1 0 log_to_monitors true 2024-03-28T01:06:05.047 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-03-28T01:06:05.045+0000 7f99eb138640 -1 osd.6 0 log_to_monitors true 2024-03-28T01:06:05.252 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-03-28T01:06:05.249+0000 7f057219a640 -1 osd.4 0 log_to_monitors true 2024-03-28T01:06:05.302 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-03-28T01:06:05.300+0000 7f0f786f9640 -1 osd.5 0 log_to_monitors true 2024-03-28T01:06:05.340 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-03-28T01:06:05.336+0000 7f6d4e0a6640 -1 osd.3 0 log_to_monitors true 2024-03-28T01:06:05.347 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-03-28T01:06:05.344+0000 7f46ef01c640 -1 osd.0 0 log_to_monitors true 2024-03-28T01:06:05.349 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-03-28T01:06:05.346+0000 7ffb5b00f640 -1 osd.2 0 log_to_monitors true 2024-03-28T01:06:05.354 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-03-28T01:06:05.352+0000 7f13e8ed7640 -1 osd.7 0 log_to_monitors true 2024-03-28T01:06:08.695 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-03-28T01:06:08.693+0000 7f1195800640 -1 osd.1 0 waiting for initial osdmap 2024-03-28T01:06:08.696 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-03-28T01:06:08.694+0000 7f99e6000640 -1 osd.6 0 waiting for initial osdmap 2024-03-28T01:06:08.705 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-03-28T01:06:08.702+0000 7f118b800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:08.705 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-03-28T01:06:08.703+0000 7f99dc000640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:08.730 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-03-28T01:06:08.728+0000 7f46e9a00640 -1 osd.0 0 waiting for initial osdmap 2024-03-28T01:06:08.736 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-03-28T01:06:08.733+0000 7f46dfa00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:08.765 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-03-28T01:06:08.763+0000 7f6d49000640 -1 osd.3 0 waiting for initial osdmap 2024-03-28T01:06:08.770 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-03-28T01:06:08.767+0000 7ffb55e00640 -1 osd.2 0 waiting for initial osdmap 2024-03-28T01:06:08.772 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-03-28T01:06:08.770+0000 7f13e3e00640 -1 osd.7 0 waiting for initial osdmap 2024-03-28T01:06:08.773 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-03-28T01:06:08.770+0000 7f6d3f000640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:08.778 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-03-28T01:06:08.775+0000 7ffb4be00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:08.779 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-03-28T01:06:08.777+0000 7f0f73600640 -1 osd.5 0 waiting for initial osdmap 2024-03-28T01:06:08.785 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-03-28T01:06:08.781+0000 7f13d9e00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:08.787 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-03-28T01:06:08.785+0000 7f0f69600640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:08.812 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-03-28T01:06:08.810+0000 7f056d000640 -1 osd.4 0 waiting for initial osdmap 2024-03-28T01:06:08.821 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-03-28T01:06:08.819+0000 7f0563000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T01:06:09.243 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:09.437 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.434+0000 7fe451e00640 1 Processor -- start 2024-03-28T01:06:09.437 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.434+0000 7fe451e00640 1 -- start start 2024-03-28T01:06:09.439 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe451e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c0b5240 0x7fe44c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:09.439 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe451e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe44c141260 0x7fe44c14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:09.439 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe451e00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44c0b5620 con 0x7fe44c1416c0 2024-03-28T01:06:09.439 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe451e00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44c0b9bc0 con 0x7fe44c141260 2024-03-28T01:06:09.439 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe451e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0b6760 con 0x7fe44c0b5240 2024-03-28T01:06:09.439 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe44be00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe44c141260 0x7fe44c14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:09.439 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe44be00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe44c141260 0x7fe44c14a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54634/0 (socket says 172.21.15.33:54634) 2024-03-28T01:06:09.440 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe44be00640 1 -- 172.21.15.33:0/1506092575 learned_addr learned my addr 172.21.15.33:0/1506092575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:09.440 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.436+0000 7fe450a00640 1 --2- 172.21.15.33:0/1506092575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c0b5240 0x7fe44c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:09.440 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.437+0000 7fe44be00640 1 -- 172.21.15.33:0/1506092575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c0b5240 msgr2=0x7fe44c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:09.440 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.437+0000 7fe44be00640 1 --2- 172.21.15.33:0/1506092575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c0b5240 0x7fe44c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.440 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.437+0000 7fe44be00640 1 -- 172.21.15.33:0/1506092575 >> v1:172.21.15.33:6789/0 conn(0x7fe44c1416c0 legacy=0x7fe44c0b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:09.440 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.437+0000 7fe44be00640 1 -- 172.21.15.33:0/1506092575 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0b9850 con 0x7fe44c141260 2024-03-28T01:06:09.441 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.438+0000 7fe44be00640 1 --2- 172.21.15.33:0/1506092575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c14a7a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe438009870 tx=0x7fe43802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848c7bbb33eec88f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:09.441 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.438+0000 7fe44b400640 1 -- 172.21.15.33:0/1506092575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438034070 con 0x7fe44c141260 2024-03-28T01:06:09.441 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.438+0000 7fe44b400640 1 -- 172.21.15.33:0/1506092575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438030840 con 0x7fe44c141260 2024-03-28T01:06:09.441 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.438+0000 7fe44b400640 1 -- 172.21.15.33:0/1506092575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438030c10 con 0x7fe44c141260 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 -- 172.21.15.33:0/1506092575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 msgr2=0x7fe44c14a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 --2- 172.21.15.33:0/1506092575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c14a7a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe438009870 tx=0x7fe43802b980 comp rx=0 tx=0).stop 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 -- 172.21.15.33:0/1506092575 shutdown_connections 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 --2- 172.21.15.33:0/1506092575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c14a7a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 --2- 172.21.15.33:0/1506092575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c0b5240 0x7fe44c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 -- 172.21.15.33:0/1506092575 >> 172.21.15.33:0/1506092575 conn(0x7fe44c0729b0 msgr2=0x7fe44c072dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 -- 172.21.15.33:0/1506092575 shutdown_connections 2024-03-28T01:06:09.442 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.439+0000 7fe451e00640 1 -- 172.21.15.33:0/1506092575 wait complete. 2024-03-28T01:06:09.443 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.440+0000 7fe451e00640 1 Processor -- start 2024-03-28T01:06:09.443 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.441+0000 7fe451e00640 1 -- start start 2024-03-28T01:06:09.445 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.441+0000 7fe451e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe44c0b5240 0x7fe44c143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:09.445 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.441+0000 7fe451e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c1e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:09.445 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.441+0000 7fe451e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c1416c0 0x7fe44c143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:09.445 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.441+0000 7fe451e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0b6d70 con 0x7fe44c141260 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.441+0000 7fe451e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0b6bf0 con 0x7fe44c0b5240 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.441+0000 7fe451e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c14ace0 con 0x7fe44c1416c0 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c1e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c1e03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54640/0 (socket says 172.21.15.33:54640) 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 -- 172.21.15.33:0/261147508 learned_addr learned my addr 172.21.15.33:0/261147508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe450a00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe44c0b5240 0x7fe44c143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe451400640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c1416c0 0x7fe44c143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 -- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c1416c0 msgr2=0x7fe44c143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c1416c0 0x7fe44c143fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 -- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe44c0b5240 msgr2=0x7fe44c143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe44c0b5240 0x7fe44c143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 -- 172.21.15.33:0/261147508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c1dfbc0 con 0x7fe44c141260 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.442+0000 7fe44be00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c1e03c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe43802b9c0 tx=0x7fe438004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:09.446 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.443+0000 7fe448c00640 1 -- 172.21.15.33:0/261147508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438048070 con 0x7fe44c141260 2024-03-28T01:06:09.447 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.443+0000 7fe448c00640 1 -- 172.21.15.33:0/261147508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438030640 con 0x7fe44c141260 2024-03-28T01:06:09.447 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.443+0000 7fe448c00640 1 -- 172.21.15.33:0/261147508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438034040 con 0x7fe44c141260 2024-03-28T01:06:09.447 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.443+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44c1dff00 con 0x7fe44c141260 2024-03-28T01:06:09.447 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.443+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c143bf0 con 0x7fe44c141260 2024-03-28T01:06:09.447 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.444+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44c075410 con 0x7fe44c141260 2024-03-28T01:06:09.447 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.445+0000 7fe448c00640 1 -- 172.21.15.33:0/261147508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7fe43804b070 con 0x7fe44c141260 2024-03-28T01:06:09.448 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.445+0000 7fe448c00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe42c06bb40 0x7fe42c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:09.448 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.446+0000 7fe448c00640 1 -- 172.21.15.33:0/261147508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fe4380abad0 con 0x7fe44c141260 2024-03-28T01:06:09.448 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.446+0000 7fe450a00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe42c06bb40 0x7fe42c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:09.449 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.446+0000 7fe450a00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe42c06bb40 0x7fe42c06e000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe444009a60 tx=0x7fe444003990 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:09.453 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.451+0000 7fe448c00640 1 -- 172.21.15.33:0/261147508 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe4380ade20 con 0x7fe44c141260 2024-03-28T01:06:09.646 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.643+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe44c067110 con 0x7fe44c141260 2024-03-28T01:06:09.647 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.645+0000 7fe448c00640 1 -- 172.21.15.33:0/261147508 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fe44c1dff00 con 0x7fe44c141260 2024-03-28T01:06:09.647 INFO:teuthology.misc.health.smithi033.stdout: 2024-03-28T01:06:09.647 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":12,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:06.410488+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:09.650 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe42c06bb40 msgr2=0x7fe42c06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:09.650 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe42c06bb40 0x7fe42c06e000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe444009a60 tx=0x7fe444003990 comp rx=0 tx=0).stop 2024-03-28T01:06:09.650 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 msgr2=0x7fe44c1e03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c1e03c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe43802b9c0 tx=0x7fe438004730 comp rx=0 tx=0).stop 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 shutdown_connections 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe42c06bb40 0x7fe42c06e000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe44c1416c0 0x7fe44c143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44c141260 0x7fe44c1e03c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 --2- 172.21.15.33:0/261147508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe44c0b5240 0x7fe44c143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 >> 172.21.15.33:0/261147508 conn(0x7fe44c0729b0 msgr2=0x7fe44c13ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 shutdown_connections 2024-03-28T01:06:09.651 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:09.647+0000 7fe451e00640 1 -- 172.21.15.33:0/261147508 wait complete. 2024-03-28T01:06:09.663 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-28T01:06:14.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:06:14.123 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:06:16.471 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:16.663 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.660+0000 7fe642800640 1 Processor -- start 2024-03-28T01:06:16.663 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.660+0000 7fe642800640 1 -- start start 2024-03-28T01:06:16.664 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.661+0000 7fe642800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fe63c0b3e00 0x7fe63c14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.661+0000 7fe642800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe63c14ace0 0x7fe63c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.661+0000 7fe642800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c0b5620 con 0x7fe63c0b5240 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe642800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c0b9bc0 con 0x7fe63c14ace0 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe642800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c141970 con 0x7fe63c0b3e00 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe641e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe63c14ace0 0x7fe63c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe641e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe63c14ace0 0x7fe63c151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58664/0 (socket says 172.21.15.33:58664) 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe641e00640 1 -- 172.21.15.33:0/2274286483 learned_addr learned my addr 172.21.15.33:0/2274286483 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe640a00640 1 --2- 172.21.15.33:0/2274286483 >> v2:172.21.15.194:3300/0 conn(0x7fe63c0b3e00 0x7fe63c14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe641e00640 1 -- 172.21.15.33:0/2274286483 >> v2:172.21.15.194:3300/0 conn(0x7fe63c0b3e00 msgr2=0x7fe63c14a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe641e00640 1 --2- 172.21.15.33:0/2274286483 >> v2:172.21.15.194:3300/0 conn(0x7fe63c0b3e00 0x7fe63c14a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe641e00640 1 -- 172.21.15.33:0/2274286483 >> v1:172.21.15.194:6789/0 conn(0x7fe63c0b5240 legacy=0x7fe63c141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:16.665 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.662+0000 7fe641e00640 1 -- 172.21.15.33:0/2274286483 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0b6760 con 0x7fe63c14ace0 2024-03-28T01:06:16.666 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.664+0000 7fe641e00640 1 --2- 172.21.15.33:0/2274286483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c151ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe63c073770 tx=0x7fe63402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eee5127a46bb16a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:16.666 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.664+0000 7fe63be00640 1 -- 172.21.15.33:0/2274286483 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63402ca80 con 0x7fe63c14ace0 2024-03-28T01:06:16.666 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.664+0000 7fe63be00640 1 -- 172.21.15.33:0/2274286483 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63402cc20 con 0x7fe63c14ace0 2024-03-28T01:06:16.667 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.664+0000 7fe63be00640 1 -- 172.21.15.33:0/2274286483 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340343f0 con 0x7fe63c14ace0 2024-03-28T01:06:16.667 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 -- 172.21.15.33:0/2274286483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 msgr2=0x7fe63c151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:16.667 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 --2- 172.21.15.33:0/2274286483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c151ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe63c073770 tx=0x7fe63402bee0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.668 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 -- 172.21.15.33:0/2274286483 shutdown_connections 2024-03-28T01:06:16.668 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 --2- 172.21.15.33:0/2274286483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c151ad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.668 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 --2- 172.21.15.33:0/2274286483 >> v2:172.21.15.194:3300/0 conn(0x7fe63c0b3e00 0x7fe63c14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.668 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 -- 172.21.15.33:0/2274286483 >> 172.21.15.33:0/2274286483 conn(0x7fe63c0729b0 msgr2=0x7fe63c072dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:16.668 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 -- 172.21.15.33:0/2274286483 shutdown_connections 2024-03-28T01:06:16.668 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.665+0000 7fe642800640 1 -- 172.21.15.33:0/2274286483 wait complete. 2024-03-28T01:06:16.670 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.667+0000 7fe642800640 1 Processor -- start 2024-03-28T01:06:16.670 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.667+0000 7fe642800640 1 -- start start 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.668+0000 7fe642800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe63c0b3e00 0x7fe63c13cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.668+0000 7fe642800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe63c0b5240 0x7fe63c136be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.668+0000 7fe642800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c137120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.668+0000 7fe642800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0b6bf0 con 0x7fe63c14ace0 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.668+0000 7fe642800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0b4420 con 0x7fe63c0b3e00 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.668+0000 7fe642800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0b6d70 con 0x7fe63c0b5240 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c137120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c137120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58680/0 (socket says 172.21.15.33:58680) 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 -- 172.21.15.33:0/371529778 learned_addr learned my addr 172.21.15.33:0/371529778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe640a00640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe63c0b5240 0x7fe63c136be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641400640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe63c0b3e00 0x7fe63c13cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:16.672 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 -- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe63c0b5240 msgr2=0x7fe63c136be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:16.673 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe63c0b5240 0x7fe63c136be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.673 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 -- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe63c0b3e00 msgr2=0x7fe63c13cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:16.673 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe63c0b3e00 0x7fe63c13cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.673 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 -- 172.21.15.33:0/371529778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c1474a0 con 0x7fe63c14ace0 2024-03-28T01:06:16.673 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641400640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe63c0b3e00 0x7fe63c13cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:16.673 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.669+0000 7fe641e00640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c137120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe63402c4f0 tx=0x7fe634004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:16.674 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.670+0000 7fe639600640 1 -- 172.21.15.33:0/371529778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634043020 con 0x7fe63c14ace0 2024-03-28T01:06:16.674 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.670+0000 7fe639600640 1 -- 172.21.15.33:0/371529778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63403e450 con 0x7fe63c14ace0 2024-03-28T01:06:16.674 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.670+0000 7fe639600640 1 -- 172.21.15.33:0/371529778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634009510 con 0x7fe63c14ace0 2024-03-28T01:06:16.674 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.670+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c147780 con 0x7fe63c14ace0 2024-03-28T01:06:16.674 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.670+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c0bc300 con 0x7fe63c14ace0 2024-03-28T01:06:16.674 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.671+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c075410 con 0x7fe63c14ace0 2024-03-28T01:06:16.675 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.673+0000 7fe639600640 1 -- 172.21.15.33:0/371529778 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe634046020 con 0x7fe63c14ace0 2024-03-28T01:06:16.676 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.674+0000 7fe639600640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe61c06bbd0 0x7fe61c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:16.677 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.674+0000 7fe639600640 1 -- 172.21.15.33:0/371529778 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe6340acf50 con 0x7fe63c14ace0 2024-03-28T01:06:16.677 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.674+0000 7fe641400640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe61c06bbd0 0x7fe61c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:16.678 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.675+0000 7fe641400640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe61c06bbd0 0x7fe61c06e090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe630003eb0 tx=0x7fe630009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:16.682 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.679+0000 7fe639600640 1 -- 172.21.15.33:0/371529778 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe63404d030 con 0x7fe63c14ace0 2024-03-28T01:06:16.888 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.886+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe63c0b61a0 con 0x7fe63c14ace0 2024-03-28T01:06:16.890 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.887+0000 7fe639600640 1 -- 172.21.15.33:0/371529778 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12884 (secure 0 0 0) 0x7fe63407bd80 con 0x7fe63c14ace0 2024-03-28T01:06:16.890 INFO:teuthology.misc.health.smithi033.stdout: 2024-03-28T01:06:16.890 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":16,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:12.729215+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe61c06bbd0 msgr2=0x7fe61c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe61c06bbd0 0x7fe61c06e090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe630003eb0 tx=0x7fe630009040 comp rx=0 tx=0).stop 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 msgr2=0x7fe63c137120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c137120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe63402c4f0 tx=0x7fe634004a40 comp rx=0 tx=0).stop 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 shutdown_connections 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe61c06bbd0 0x7fe61c06e090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c14ace0 0x7fe63c137120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe63c0b5240 0x7fe63c136be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.893 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 --2- 172.21.15.33:0/371529778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe63c0b3e00 0x7fe63c13cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:16.894 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 >> 172.21.15.33:0/371529778 conn(0x7fe63c0729b0 msgr2=0x7fe63c13e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:16.894 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.890+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 shutdown_connections 2024-03-28T01:06:16.894 INFO:teuthology.misc.health.smithi033.stderr:2024-03-28T01:06:16.891+0000 7fe642800640 1 -- 172.21.15.33:0/371529778 wait complete. 2024-03-28T01:06:16.908 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-28T01:06:16.908 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-28T01:06:16.908 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T01:06:16.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T01:06:16.913 INFO:tasks.ceph.mds.a:Started 2024-03-28T01:06:16.913 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T01:06:16.914 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T01:06:16.921 INFO:tasks.ceph.mds.b:Started 2024-03-28T01:06:16.921 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-28T01:06:16.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:17.075 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2024-03-28T01:06:17.101 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-03-28T01:06:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.109+0000 7f2849c00640 1 Processor -- start 2024-03-28T01:06:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.109+0000 7f2849c00640 1 -- start start 2024-03-28T01:06:17.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2849c00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f28440b51d0 0x7f28440b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2849c00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28440b8d00 0x7f284414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2849c00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28440b55b0 con 0x7f28440b42f0 2024-03-28T01:06:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2849c00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284414a820 con 0x7f28440b8d00 2024-03-28T01:06:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2849c00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28440b9190 con 0x7f28440b51d0 2024-03-28T01:06:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2843400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28440b8d00 0x7f284414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2849200640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f28440b42f0 0x7f284414b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56380/0 (socket says 172.21.15.33:56380) 2024-03-28T01:06:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.110+0000 7f2849200640 1 -- 172.21.15.33:0/3326145724 learned_addr learned my addr 172.21.15.33:0/3326145724 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.111+0000 7f2843e00640 1 --2- 172.21.15.33:0/3326145724 >> v2:172.21.15.194:3300/0 conn(0x7f28440b51d0 0x7f28440b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.111+0000 7f2843400640 1 -- 172.21.15.33:0/3326145724 >> v2:172.21.15.194:3300/0 conn(0x7f28440b51d0 msgr2=0x7f28440b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.111+0000 7f2843400640 1 --2- 172.21.15.33:0/3326145724 >> v2:172.21.15.194:3300/0 conn(0x7f28440b51d0 0x7f28440b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.111+0000 7f2843400640 1 -- 172.21.15.33:0/3326145724 >> v1:172.21.15.33:6789/0 conn(0x7f28440b42f0 legacy=0x7f284414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.111+0000 7f2843400640 1 -- 172.21.15.33:0/3326145724 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440b9800 con 0x7f28440b8d00 2024-03-28T01:06:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.111+0000 7f2842a00640 1 -- 172.21.15.33:0/3326145724 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152897875 0 0) 0x7f28440b55b0 con 0x7f28440b42f0 2024-03-28T01:06:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.111+0000 7f2843e00640 1 --2- 172.21.15.33:0/3326145724 >> v2:172.21.15.194:3300/0 conn(0x7f28440b51d0 0x7f28440b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.112+0000 7f2843400640 1 --2- 172.21.15.33:0/3326145724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28440b8d00 0x7f284414ac70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28300097a0 tx=0x7f283002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bef1df86f76383b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.112+0000 7f2842a00640 1 -- 172.21.15.33:0/3326145724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830035020 con 0x7f28440b8d00 2024-03-28T01:06:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.112+0000 7f2842a00640 1 -- 172.21.15.33:0/3326145724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2830031830 con 0x7f28440b8d00 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.112+0000 7f2842a00640 1 -- 172.21.15.33:0/3326145724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830031c00 con 0x7f28440b8d00 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.112+0000 7f2849c00640 1 -- 172.21.15.33:0/3326145724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28440b8d00 msgr2=0x7f284414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.112+0000 7f2849c00640 1 --2- 172.21.15.33:0/3326145724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28440b8d00 0x7f284414ac70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28300097a0 tx=0x7f283002b8d0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.113+0000 7f2849c00640 1 -- 172.21.15.33:0/3326145724 shutdown_connections 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.113+0000 7f2849c00640 1 --2- 172.21.15.33:0/3326145724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28440b8d00 0x7f284414ac70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.113+0000 7f2849c00640 1 --2- 172.21.15.33:0/3326145724 >> v2:172.21.15.194:3300/0 conn(0x7f28440b51d0 0x7f28440b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.113+0000 7f2849c00640 1 -- 172.21.15.33:0/3326145724 >> 172.21.15.33:0/3326145724 conn(0x7f2844072940 msgr2=0x7f2844072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.113+0000 7f2849c00640 1 -- 172.21.15.33:0/3326145724 shutdown_connections 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.113+0000 7f2849c00640 1 -- 172.21.15.33:0/3326145724 wait complete. 2024-03-28T01:06:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.114+0000 7f2849c00640 1 Processor -- start 2024-03-28T01:06:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.114+0000 7f2849c00640 1 -- start start 2024-03-28T01:06:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.115+0000 7f2849c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28440b42f0 0x7f28440aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.115+0000 7f2849c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28440b51d0 0x7f28440aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.115+0000 7f2849c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2844143540 0x7f28441439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.115+0000 7f2849c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b6b80 con 0x7f2844143540 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.115+0000 7f2849c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b39c0 con 0x7f28440b42f0 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.115+0000 7f2849c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b6d00 con 0x7f28440b51d0 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.115+0000 7f2849200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2844143540 0x7f28441439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2844143540 0x7f28441439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58706/0 (socket says 172.21.15.33:58706) 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 -- 172.21.15.33:0/2922654194 learned_addr learned my addr 172.21.15.33:0/2922654194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2843e00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28440b42f0 0x7f28440aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2843400640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28440b51d0 0x7f28440aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 -- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28440b51d0 msgr2=0x7f28440aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28440b51d0 0x7f28440aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 -- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28440b42f0 msgr2=0x7f28440aa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28440b42f0 0x7f28440aa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 -- 172.21.15.33:0/2922654194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440ab6d0 con 0x7f2844143540 2024-03-28T01:06:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.116+0000 7f2849200640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2844143540 0x7f28441439f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c7b0 tx=0x7f283c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.117+0000 7f283be00640 1 -- 172.21.15.33:0/2922654194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c018070 con 0x7f2844143540 2024-03-28T01:06:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.117+0000 7f283be00640 1 -- 172.21.15.33:0/2922654194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c00a640 con 0x7f2844143540 2024-03-28T01:06:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.117+0000 7f283be00640 1 -- 172.21.15.33:0/2922654194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c00aa70 con 0x7f2844143540 2024-03-28T01:06:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.117+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28440ac950 con 0x7f2844143540 2024-03-28T01:06:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.117+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28441e04c0 con 0x7f2844143540 2024-03-28T01:06:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.118+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2844075310 con 0x7f2844143540 2024-03-28T01:06:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.119+0000 7f283be00640 1 -- 172.21.15.33:0/2922654194 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f283c00ac10 con 0x7f2844143540 2024-03-28T01:06:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.120+0000 7f283be00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f282406bc00 0x7f282406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.120+0000 7f283be00640 1 -- 172.21.15.33:0/2922654194 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f283c089ca0 con 0x7f2844143540 2024-03-28T01:06:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.120+0000 7f2843e00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f282406bc00 0x7f282406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.121+0000 7f2843e00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f282406bc00 0x7f282406e0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f283400ab60 tx=0x7f2834008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.125+0000 7f283be00640 1 -- 172.21.15.33:0/2922654194 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f283c0583f0 con 0x7f2844143540 2024-03-28T01:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.327+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2844145310 con 0x7f2844143540 2024-03-28T01:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.329+0000 7f283be00640 1 -- 172.21.15.33:0/2922654194 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f283c05c490 con 0x7f2844143540 2024-03-28T01:06:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:17.332 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:12.729215+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.331+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f282406bc00 msgr2=0x7f282406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.331+0000 7f2849c00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f282406bc00 0x7f282406e0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f283400ab60 tx=0x7f2834008040 comp rx=0 tx=0).stop 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.331+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2844143540 msgr2=0x7f28441439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.331+0000 7f2849c00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2844143540 0x7f28441439f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c7b0 tx=0x7f283c00cc70 comp rx=0 tx=0).stop 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 shutdown_connections 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f282406bc00 0x7f282406e0c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2844143540 0x7f28441439f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28440b51d0 0x7f28440aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 --2- 172.21.15.33:0/2922654194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28440b42f0 0x7f28440aa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 >> 172.21.15.33:0/2922654194 conn(0x7f2844072940 msgr2=0x7f28441400a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 shutdown_connections 2024-03-28T01:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.332+0000 7f2849c00640 1 -- 172.21.15.33:0/2922654194 wait complete. 2024-03-28T01:06:17.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T01:06:10.462126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T01:06:17.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T01:06:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.571+0000 7fcd0fc00640 1 Processor -- start 2024-03-28T01:06:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.572+0000 7fcd0fc00640 1 -- start start 2024-03-28T01:06:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.572+0000 7fcd0fc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd080b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0fc00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd081476e0 0x7fcd08141340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0fc00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd080ba2c0 con 0x7fcd08147260 2024-03-28T01:06:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0fc00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd080b5ab0 con 0x7fcd081476e0 2024-03-28T01:06:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0fc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b6900 con 0x7fcd080b9ee0 2024-03-28T01:06:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0dc00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd081476e0 0x7fcd08141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd07e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcd08147260 0x7fcd08140c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56388/0 (socket says 172.21.15.33:56388) 2024-03-28T01:06:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd07e00640 1 -- 172.21.15.33:0/3965709654 learned_addr learned my addr 172.21.15.33:0/3965709654 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0d200640 1 --2- 172.21.15.33:0/3965709654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd080b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd07400640 1 -- 172.21.15.33:0/3965709654 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214057641 0 0) 0x7fcd080ba2c0 con 0x7fcd08147260 2024-03-28T01:06:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0dc00640 1 -- 172.21.15.33:0/3965709654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 msgr2=0x7fcd080b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0dc00640 1 --2- 172.21.15.33:0/3965709654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd080b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0dc00640 1 -- 172.21.15.33:0/3965709654 >> v1:172.21.15.33:6789/0 conn(0x7fcd08147260 legacy=0x7fcd08140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.573+0000 7fcd0dc00640 1 -- 172.21.15.33:0/3965709654 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080b4a00 con 0x7fcd081476e0 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd0dc00640 1 --2- 172.21.15.33:0/3965709654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd081476e0 0x7fcd08141340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009d90 tx=0x7fcd0002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d407dd7f50a4818 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd07400640 1 -- 172.21.15.33:0/3965709654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00039070 con 0x7fcd081476e0 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd07400640 1 -- 172.21.15.33:0/3965709654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0002cc50 con 0x7fcd081476e0 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd07400640 1 -- 172.21.15.33:0/3965709654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00034440 con 0x7fcd081476e0 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3965709654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd081476e0 msgr2=0x7fcd08141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3965709654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd081476e0 0x7fcd08141340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009d90 tx=0x7fcd0002bea0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3965709654 shutdown_connections 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3965709654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd081476e0 0x7fcd08141340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3965709654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd080b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.575+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3965709654 >> 172.21.15.33:0/3965709654 conn(0x7fcd08072b80 msgr2=0x7fcd08072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.576+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3965709654 shutdown_connections 2024-03-28T01:06:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.576+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3965709654 wait complete. 2024-03-28T01:06:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.577+0000 7fcd0fc00640 1 Processor -- start 2024-03-28T01:06:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.577+0000 7fcd0fc00640 1 -- start start 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd0fc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd081d3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd0fc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd08147260 0x7fcd081cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd0fc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd081476e0 0x7fcd081cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd0fc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b44e0 con 0x7fcd08147260 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd0fc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b4200 con 0x7fcd081476e0 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd0fc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b7070 con 0x7fcd080b9ee0 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd07e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd08147260 0x7fcd081cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd07e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd08147260 0x7fcd081cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58734/0 (socket says 172.21.15.33:58734) 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd07e00640 1 -- 172.21.15.33:0/3812556303 learned_addr learned my addr 172.21.15.33:0/3812556303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.578+0000 7fcd0d200640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd081d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd0dc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd081476e0 0x7fcd081cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd07e00640 1 -- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 msgr2=0x7fcd081d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd07e00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd081d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd07e00640 1 -- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd081476e0 msgr2=0x7fcd081cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd07e00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd081476e0 0x7fcd081cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd07e00640 1 -- 172.21.15.33:0/3812556303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd081dcfd0 con 0x7fcd08147260 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd0d200640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd081d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd0dc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd081476e0 0x7fcd081cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd07e00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd08147260 0x7fcd081cd340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c4d0 tx=0x7fccf400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd04c00640 1 -- 172.21.15.33:0/3812556303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4018070 con 0x7fcd08147260 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd04c00640 1 -- 172.21.15.33:0/3812556303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4007500 con 0x7fcd08147260 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd081dd2b0 con 0x7fcd08147260 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd04c00640 1 -- 172.21.15.33:0/3812556303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4004a20 con 0x7fcd08147260 2024-03-28T01:06:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.579+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd081d27d0 con 0x7fcd08147260 2024-03-28T01:06:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.580+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd08075310 con 0x7fcd08147260 2024-03-28T01:06:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.582+0000 7fcd04c00640 1 -- 172.21.15.33:0/3812556303 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fccf401b070 con 0x7fcd08147260 2024-03-28T01:06:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.582+0000 7fcd04c00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcce806bbd0 0x7fcce806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.582+0000 7fcd04c00640 1 -- 172.21.15.33:0/3812556303 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fccf4089a40 con 0x7fcd08147260 2024-03-28T01:06:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.583+0000 7fcd0d200640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcce806bbd0 0x7fcce806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.583+0000 7fcd0d200640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcce806bbd0 0x7fcce806e090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fccf8008890 tx=0x7fccf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.587+0000 7fcd04c00640 1 -- 172.21.15.33:0/3812556303 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fccf4058870 con 0x7fcd08147260 2024-03-28T01:06:17.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.793+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd080bc450 con 0x7fcd08147260 2024-03-28T01:06:17.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.795+0000 7fcd04c00640 1 -- 172.21.15.33:0/3812556303 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccf4058170 con 0x7fcd08147260 2024-03-28T01:06:17.797 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-03-28T01:06:17.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.797+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcce806bbd0 msgr2=0x7fcce806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.797+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcce806bbd0 0x7fcce806e090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fccf8008890 tx=0x7fccf8008040 comp rx=0 tx=0).stop 2024-03-28T01:06:17.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.797+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd08147260 msgr2=0x7fcd081cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:17.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.797+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd08147260 0x7fcd081cd340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c4d0 tx=0x7fccf400c990 comp rx=0 tx=0).stop 2024-03-28T01:06:17.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 shutdown_connections 2024-03-28T01:06:17.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcce806bbd0 0x7fcce806e090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd081476e0 0x7fcd081cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd08147260 0x7fcd081cd340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 --2- 172.21.15.33:0/3812556303 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd080b9ee0 0x7fcd081d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 >> 172.21.15.33:0/3812556303 conn(0x7fcd08072b80 msgr2=0x7fcd080b7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 shutdown_connections 2024-03-28T01:06:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:17.798+0000 7fcd0fc00640 1 -- 172.21.15.33:0/3812556303 wait complete. 2024-03-28T01:06:17.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T01:06:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.036+0000 7fcd80a00640 1 Processor -- start 2024-03-28T01:06:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.036+0000 7fcd80a00640 1 -- start start 2024-03-28T01:06:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.037+0000 7fcd80a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd7c0b5550 0x7fcd7c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.037+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.037+0000 7fcd80a00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c0b0e70 con 0x7fcd7c0727d0 2024-03-28T01:06:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.037+0000 7fcd80a00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b5930 con 0x7fcd7c0b5550 2024-03-28T01:06:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.037+0000 7fcd80a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b1110 con 0x7fcd7c141750 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7b400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd7c0b5550 0x7fcd7c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7b400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd7c0b5550 0x7fcd7c071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58738/0 (socket says 172.21.15.33:58738) 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7b400640 1 -- 172.21.15.33:0/3817269482 learned_addr learned my addr 172.21.15.33:0/3817269482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7be00640 1 --2- 172.21.15.33:0/3817269482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7b400640 1 -- 172.21.15.33:0/3817269482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 msgr2=0x7fcd7c14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7b400640 1 --2- 172.21.15.33:0/3817269482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7b400640 1 -- 172.21.15.33:0/3817269482 >> v1:172.21.15.194:6789/0 conn(0x7fcd7c0727d0 legacy=0x7fcd7c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.038+0000 7fcd7b400640 1 -- 172.21.15.33:0/3817269482 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0b13f0 con 0x7fcd7c0b5550 2024-03-28T01:06:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.039+0000 7fcd7b400640 1 --2- 172.21.15.33:0/3817269482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c071a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009870 tx=0x7fcd6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b0e8521406d25c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd7aa00640 1 -- 172.21.15.33:0/3817269482 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c035020 con 0x7fcd7c0b5550 2024-03-28T01:06:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd7aa00640 1 -- 172.21.15.33:0/3817269482 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c031640 con 0x7fcd7c0b5550 2024-03-28T01:06:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd7aa00640 1 -- 172.21.15.33:0/3817269482 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c02d040 con 0x7fcd7c0b5550 2024-03-28T01:06:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd80a00640 1 -- 172.21.15.33:0/3817269482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 msgr2=0x7fcd7c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd80a00640 1 --2- 172.21.15.33:0/3817269482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c071a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009870 tx=0x7fcd6c02b8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd80a00640 1 -- 172.21.15.33:0/3817269482 shutdown_connections 2024-03-28T01:06:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd80a00640 1 --2- 172.21.15.33:0/3817269482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd80a00640 1 --2- 172.21.15.33:0/3817269482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c071a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.040+0000 7fcd80a00640 1 -- 172.21.15.33:0/3817269482 >> 172.21.15.33:0/3817269482 conn(0x7fcd7c0b63d0 msgr2=0x7fcd7c0b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.041+0000 7fcd80a00640 1 -- 172.21.15.33:0/3817269482 shutdown_connections 2024-03-28T01:06:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.041+0000 7fcd80a00640 1 -- 172.21.15.33:0/3817269482 wait complete. 2024-03-28T01:06:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.042+0000 7fcd80a00640 1 Processor -- start 2024-03-28T01:06:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.042+0000 7fcd80a00640 1 -- start start 2024-03-28T01:06:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd7c0727d0 0x7fcd7c1dcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd80a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c14aed0 con 0x7fcd7c0b5550 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd80a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c14ad50 con 0x7fcd7c0727d0 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd80a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c14b050 con 0x7fcd7c141750 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd73e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd73e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c1470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58744/0 (socket says 172.21.15.33:58744) 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.043+0000 7fcd73e00640 1 -- 172.21.15.33:0/1346099467 learned_addr learned my addr 172.21.15.33:0/1346099467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd7b400640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd7c0727d0 0x7fcd7c1dcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73e00640 1 -- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 msgr2=0x7fcd7c145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd7be00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73e00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73e00640 1 -- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd7c0727d0 msgr2=0x7fcd7c1dcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73e00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd7c0727d0 0x7fcd7c1dcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73e00640 1 -- 172.21.15.33:0/1346099467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0b0700 con 0x7fcd7c0b5550 2024-03-28T01:06:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd7b400640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd7c0727d0 0x7fcd7c1dcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73e00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c1470d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400c3c0 tx=0x7fcd6400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73400640 1 -- 172.21.15.33:0/1346099467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640052d0 con 0x7fcd7c0b5550 2024-03-28T01:06:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73400640 1 -- 172.21.15.33:0/1346099467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd64005490 con 0x7fcd7c0b5550 2024-03-28T01:06:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c147610 con 0x7fcd7c0b5550 2024-03-28T01:06:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.044+0000 7fcd73400640 1 -- 172.21.15.33:0/1346099467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64007260 con 0x7fcd7c0b5550 2024-03-28T01:06:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.045+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c146bf0 con 0x7fcd7c0b5550 2024-03-28T01:06:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.046+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c0b1960 con 0x7fcd7c0b5550 2024-03-28T01:06:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.046+0000 7fcd73400640 1 -- 172.21.15.33:0/1346099467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcd64007400 con 0x7fcd7c0b5550 2024-03-28T01:06:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.047+0000 7fcd73400640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd5806bbd0 0x7fcd5806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.047+0000 7fcd73400640 1 -- 172.21.15.33:0/1346099467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd64089a90 con 0x7fcd7c0b5550 2024-03-28T01:06:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.048+0000 7fcd7b400640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd5806bbd0 0x7fcd5806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.048+0000 7fcd7b400640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd5806bbd0 0x7fcd5806e090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c036000 tx=0x7fcd6c03c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.052+0000 7fcd73400640 1 -- 172.21.15.33:0/1346099467 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fcd7c0b1960 con 0x7fcd7c0b5550 2024-03-28T01:06:18.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.295+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd7c146df0 con 0x7fcd7c0b5550 2024-03-28T01:06:18.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.296+0000 7fcd73400640 1 -- 172.21.15.33:0/1346099467 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2114 (secure 0 0 0) 0x7fcd640588c0 con 0x7fcd7c0b5550 2024-03-28T01:06:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:18.299 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/214301400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":214301400},{"type":"v1","addr":"172.21.15.194:6833","nonce":214301400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2290842450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2290842450},{"type":"v1","addr":"172.21.15.33:6835","nonce":2290842450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-28T01:06:18.299 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd5806bbd0 msgr2=0x7fcd5806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd5806bbd0 0x7fcd5806e090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c036000 tx=0x7fcd6c03c040 comp rx=0 tx=0).stop 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 msgr2=0x7fcd7c1470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c1470d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400c3c0 tx=0x7fcd6400c880 comp rx=0 tx=0).stop 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 shutdown_connections 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd5806bbd0 0x7fcd5806e090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd7c141750 0x7fcd7c145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7c0b5550 0x7fcd7c1470d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 --2- 172.21.15.33:0/1346099467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd7c0727d0 0x7fcd7c1dcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 >> 172.21.15.33:0/1346099467 conn(0x7fcd7c0b63d0 msgr2=0x7fcd7c13e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 shutdown_connections 2024-03-28T01:06:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.299+0000 7fcd80a00640 1 -- 172.21.15.33:0/1346099467 wait complete. 2024-03-28T01:06:18.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.522+0000 7f170da00640 1 Processor -- start 2024-03-28T01:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.522+0000 7f170da00640 1 -- start start 2024-03-28T01:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.524+0000 7f170da00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f17080b6b80 0x7f170814b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.524+0000 7f170da00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17080b55b0 con 0x7f17080b51d0 2024-03-28T01:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.524+0000 7f170da00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17080b9190 con 0x7f17080b8d00 2024-03-28T01:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.524+0000 7f170da00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170814a820 con 0x7f17080b6b80 2024-03-28T01:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.524+0000 7f1707e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f17080b51d0 0x7f17080b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56400/0 (socket says 172.21.15.33:56400) 2024-03-28T01:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.524+0000 7f1707e00640 1 -- 172.21.15.33:0/2954308721 learned_addr learned my addr 172.21.15.33:0/2954308721 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.524+0000 7f170d000640 1 --2- 172.21.15.33:0/2954308721 >> v2:172.21.15.33:3300/0 conn(0x7f17080b6b80 0x7f170814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.525+0000 7f170d000640 1 -- 172.21.15.33:0/2954308721 >> v1:172.21.15.194:6789/0 conn(0x7f17080b8d00 legacy=0x7f170814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.525+0000 7f170d000640 1 -- 172.21.15.33:0/2954308721 >> v1:172.21.15.33:6789/0 conn(0x7f17080b51d0 legacy=0x7f17080b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.525+0000 7f170d000640 1 -- 172.21.15.33:0/2954308721 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080b9800 con 0x7f17080b6b80 2024-03-28T01:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.525+0000 7f1706a00640 1 -- 172.21.15.33:0/2954308721 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936842993 0 0) 0x7f17080b55b0 con 0x7f17080b51d0 2024-03-28T01:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.526+0000 7f170d000640 1 --2- 172.21.15.33:0/2954308721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f170814b380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1700009d90 tx=0x7f170002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4828ef15ce2e6d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.526+0000 7f1706a00640 1 -- 172.21.15.33:0/2954308721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700039070 con 0x7f17080b6b80 2024-03-28T01:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.526+0000 7f1706a00640 1 -- 172.21.15.33:0/2954308721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170002cd00 con 0x7f17080b6b80 2024-03-28T01:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.526+0000 7f1706a00640 1 -- 172.21.15.33:0/2954308721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700034660 con 0x7f17080b6b80 2024-03-28T01:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.526+0000 7f170da00640 1 -- 172.21.15.33:0/2954308721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 msgr2=0x7f170814b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.526+0000 7f170da00640 1 --2- 172.21.15.33:0/2954308721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f170814b380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1700009d90 tx=0x7f170002bf50 comp rx=0 tx=0).stop 2024-03-28T01:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.527+0000 7f170da00640 1 -- 172.21.15.33:0/2954308721 shutdown_connections 2024-03-28T01:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.527+0000 7f170da00640 1 --2- 172.21.15.33:0/2954308721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f170814b380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.527+0000 7f170da00640 1 -- 172.21.15.33:0/2954308721 >> 172.21.15.33:0/2954308721 conn(0x7f1708072940 msgr2=0x7f1708072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.527+0000 7f170da00640 1 -- 172.21.15.33:0/2954308721 shutdown_connections 2024-03-28T01:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.527+0000 7f170da00640 1 -- 172.21.15.33:0/2954308721 wait complete. 2024-03-28T01:06:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.528+0000 7f170da00640 1 Processor -- start 2024-03-28T01:06:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.528+0000 7f170da00640 1 -- start start 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f170da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17080b51d0 0x7f170813c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f170da00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f17081dd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f170da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17081dd670 0x7f1708138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f170da00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17080b39c0 con 0x7f17080b6b80 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f170da00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17080b44c0 con 0x7f17081dd670 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f170da00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1708152b80 con 0x7f17080b51d0 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f1707400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f17081dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f1707400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f17081dd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58758/0 (socket says 172.21.15.33:58758) 2024-03-28T01:06:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.529+0000 7f1707400640 1 -- 172.21.15.33:0/4091206267 learned_addr learned my addr 172.21.15.33:0/4091206267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f170d000640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17081dd670 0x7f1708138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707e00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17080b51d0 0x7f170813c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707400640 1 -- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17080b51d0 msgr2=0x7f170813c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707400640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17080b51d0 0x7f170813c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707400640 1 -- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17081dd670 msgr2=0x7f1708138b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707400640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17081dd670 0x7f1708138b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707400640 1 -- 172.21.15.33:0/4091206267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708140c90 con 0x7f17080b6b80 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f170d000640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17081dd670 0x7f1708138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707e00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17080b51d0 0x7f170813c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f1707400640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f17081dd130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16f400c430 tx=0x7f16f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f16ffe00640 1 -- 172.21.15.33:0/4091206267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4017070 con 0x7f17080b6b80 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f16ffe00640 1 -- 172.21.15.33:0/4091206267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f40052d0 con 0x7f17080b6b80 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f16ffe00640 1 -- 172.21.15.33:0/4091206267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4003830 con 0x7f17080b6b80 2024-03-28T01:06:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1708140520 con 0x7f17080b6b80 2024-03-28T01:06:18.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.530+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170813fd20 con 0x7f17080b6b80 2024-03-28T01:06:18.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.531+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1708075310 con 0x7f17080b6b80 2024-03-28T01:06:18.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.532+0000 7f16ffe00640 1 -- 172.21.15.33:0/4091206267 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f16f401a070 con 0x7f17080b6b80 2024-03-28T01:06:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.533+0000 7f16ffe00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f16e806bbd0 0x7f16e806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.533+0000 7f16ffe00640 1 -- 172.21.15.33:0/4091206267 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f16f4089a70 con 0x7f17080b6b80 2024-03-28T01:06:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.533+0000 7f1707e00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f16e806bbd0 0x7f16e806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.534+0000 7f1707e00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f16e806bbd0 0x7f16e806e090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16f8000f60 tx=0x7f16f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.538+0000 7f16ffe00640 1 -- 172.21.15.33:0/4091206267 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f16f40588a0 con 0x7f17080b6b80 2024-03-28T01:06:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.740+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1708074720 con 0x7f17080b6b80 2024-03-28T01:06:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.742+0000 7f16ffe00640 1 -- 172.21.15.33:0/4091206267 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f16f40581a0 con 0x7f17080b6b80 2024-03-28T01:06:18.745 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:18.745 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:12.729215+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:18.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.744+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f16e806bbd0 msgr2=0x7f16e806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.744+0000 7f170da00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f16e806bbd0 0x7f16e806e090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16f8000f60 tx=0x7f16f8009040 comp rx=0 tx=0).stop 2024-03-28T01:06:18.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.744+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 msgr2=0x7f17081dd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.744+0000 7f170da00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f17081dd130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16f400c430 tx=0x7f16f400c8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 shutdown_connections 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f16e806bbd0 0x7f16e806e090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17081dd670 0x7f1708138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b6b80 0x7f17081dd130 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 --2- 172.21.15.33:0/4091206267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17080b51d0 0x7f170813c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 >> 172.21.15.33:0/4091206267 conn(0x7f1708072940 msgr2=0x7f17081493e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 shutdown_connections 2024-03-28T01:06:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.745+0000 7f170da00640 1 -- 172.21.15.33:0/4091206267 wait complete. 2024-03-28T01:06:18.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T01:06:10.462126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T01:06:18.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T01:06:18.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.986+0000 7fbb73000640 1 Processor -- start 2024-03-28T01:06:18.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.986+0000 7fbb73000640 1 -- start start 2024-03-28T01:06:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.988+0000 7fbb73000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c147220 0x7fbb6c147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.988+0000 7fbb73000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c0b6a90 con 0x7fbb6c141070 2024-03-28T01:06:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.988+0000 7fbb73000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c0b41b0 con 0x7fbb6c0b5500 2024-03-28T01:06:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.988+0000 7fbb73000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0ba1c0 con 0x7fbb6c147220 2024-03-28T01:06:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.988+0000 7fbb71000640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbb6c141070 0x7fbb6c151ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56414/0 (socket says 172.21.15.33:56414) 2024-03-28T01:06:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.988+0000 7fbb71000640 1 -- 172.21.15.33:0/2008600905 learned_addr learned my addr 172.21.15.33:0/2008600905 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.989+0000 7fbb6b400640 1 --2- 172.21.15.33:0/2008600905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c147220 0x7fbb6c147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.989+0000 7fbb6aa00640 1 -- 172.21.15.33:0/2008600905 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414111725 0 0) 0x7fbb6c0b6a90 con 0x7fbb6c141070 2024-03-28T01:06:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.989+0000 7fbb6aa00640 1 -- 172.21.15.33:0/2008600905 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb50003520 con 0x7fbb6c141070 2024-03-28T01:06:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.989+0000 7fbb6b400640 1 -- 172.21.15.33:0/2008600905 >> v1:172.21.15.194:6789/0 conn(0x7fbb6c0b5500 legacy=0x7fbb6c0b9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.989+0000 7fbb6aa00640 1 -- 172.21.15.33:0/2008600905 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035416001 0 0) 0x7fbb6c0b41b0 con 0x7fbb6c0b5500 2024-03-28T01:06:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.989+0000 7fbb6b400640 1 -- 172.21.15.33:0/2008600905 >> v1:172.21.15.33:6789/0 conn(0x7fbb6c141070 legacy=0x7fbb6c151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.989+0000 7fbb6b400640 1 -- 172.21.15.33:0/2008600905 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0b58e0 con 0x7fbb6c147220 2024-03-28T01:06:18.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.991+0000 7fbb6b400640 1 --2- 172.21.15.33:0/2008600905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c147220 0x7fbb6c147660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097d0 tx=0x7fbb5802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d0189b6abebd9e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:18.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.991+0000 7fbb6aa00640 1 -- 172.21.15.33:0/2008600905 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58035020 con 0x7fbb6c147220 2024-03-28T01:06:18.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.992+0000 7fbb6aa00640 1 -- 172.21.15.33:0/2008600905 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58031810 con 0x7fbb6c147220 2024-03-28T01:06:18.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.992+0000 7fbb6aa00640 1 -- 172.21.15.33:0/2008600905 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031be0 con 0x7fbb6c147220 2024-03-28T01:06:18.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.992+0000 7fbb73000640 1 -- 172.21.15.33:0/2008600905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c147220 msgr2=0x7fbb6c147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:18.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.992+0000 7fbb73000640 1 --2- 172.21.15.33:0/2008600905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c147220 0x7fbb6c147660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097d0 tx=0x7fbb5802b8b0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.993+0000 7fbb73000640 1 -- 172.21.15.33:0/2008600905 shutdown_connections 2024-03-28T01:06:18.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.993+0000 7fbb73000640 1 --2- 172.21.15.33:0/2008600905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c147220 0x7fbb6c147660 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:18.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.993+0000 7fbb73000640 1 -- 172.21.15.33:0/2008600905 >> 172.21.15.33:0/2008600905 conn(0x7fbb6c072940 msgr2=0x7fbb6c072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:18.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.993+0000 7fbb73000640 1 -- 172.21.15.33:0/2008600905 shutdown_connections 2024-03-28T01:06:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.993+0000 7fbb73000640 1 -- 172.21.15.33:0/2008600905 wait complete. 2024-03-28T01:06:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.994+0000 7fbb73000640 1 Processor -- start 2024-03-28T01:06:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.995+0000 7fbb73000640 1 -- start start 2024-03-28T01:06:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb73000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb6c0b5500 0x7fbb6c13edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb73000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c141070 0x7fbb6c1e1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb73000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb6c147220 0x7fbb6c138970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb73000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b7030 con 0x7fbb6c0b5500 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb73000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b6eb0 con 0x7fbb6c147220 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb73000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c141500 con 0x7fbb6c141070 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb6c0b5500 0x7fbb6c13edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb6c0b5500 0x7fbb6c13edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58770/0 (socket says 172.21.15.33:58770) 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 -- 172.21.15.33:0/918494019 learned_addr learned my addr 172.21.15.33:0/918494019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 -- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c141070 msgr2=0x7fbb6c1e1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6b400640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c141070 0x7fbb6c1e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c141070 0x7fbb6c1e1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 -- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb6c147220 msgr2=0x7fbb6c138970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb71000640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb6c147220 0x7fbb6c138970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb6c147220 0x7fbb6c138970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.996+0000 7fbb6be00640 1 -- 172.21.15.33:0/918494019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c139630 con 0x7fbb6c0b5500 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.997+0000 7fbb6be00640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb6c0b5500 0x7fbb6c13edc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c7e0 tx=0x7fbb5c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.997+0000 7fbb63e00640 1 -- 172.21.15.33:0/918494019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c003da0 con 0x7fbb6c0b5500 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.997+0000 7fbb63e00640 1 -- 172.21.15.33:0/918494019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c005070 con 0x7fbb6c0b5500 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.997+0000 7fbb63e00640 1 -- 172.21.15.33:0/918494019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c005460 con 0x7fbb6c0b5500 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.997+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c1398d0 con 0x7fbb6c0b5500 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.997+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c1e1560 con 0x7fbb6c0b5500 2024-03-28T01:06:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.998+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c075310 con 0x7fbb6c0b5500 2024-03-28T01:06:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:18.999+0000 7fbb63e00640 1 -- 172.21.15.33:0/918494019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbb5c005600 con 0x7fbb6c0b5500 2024-03-28T01:06:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.000+0000 7fbb63e00640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbb4c06bbd0 0x7fbb4c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.000+0000 7fbb63e00640 1 -- 172.21.15.33:0/918494019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb5c08a770 con 0x7fbb6c0b5500 2024-03-28T01:06:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.000+0000 7fbb6b400640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbb4c06bbd0 0x7fbb4c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.001+0000 7fbb6b400640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbb4c06bbd0 0x7fbb4c06e090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb58036000 tx=0x7fbb58008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.005+0000 7fbb63e00640 1 -- 172.21.15.33:0/918494019 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbb5c0595a0 con 0x7fbb6c0b5500 2024-03-28T01:06:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.207+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb6c0bc7c0 con 0x7fbb6c0b5500 2024-03-28T01:06:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.208+0000 7fbb63e00640 1 -- 172.21.15.33:0/918494019 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb5c058ea0 con 0x7fbb6c0b5500 2024-03-28T01:06:19.211 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbb4c06bbd0 msgr2=0x7fbb4c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbb4c06bbd0 0x7fbb4c06e090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb58036000 tx=0x7fbb58008030 comp rx=0 tx=0).stop 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb6c0b5500 msgr2=0x7fbb6c13edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb6c0b5500 0x7fbb6c13edc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c7e0 tx=0x7fbb5c00cca0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 shutdown_connections 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbb4c06bbd0 0x7fbb4c06e090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb6c147220 0x7fbb6c138970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb6c141070 0x7fbb6c1e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 --2- 172.21.15.33:0/918494019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb6c0b5500 0x7fbb6c13edc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 >> 172.21.15.33:0/918494019 conn(0x7fbb6c072940 msgr2=0x7fbb6c073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 shutdown_connections 2024-03-28T01:06:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.211+0000 7fbb73000640 1 -- 172.21.15.33:0/918494019 wait complete. 2024-03-28T01:06:19.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T01:06:19.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.451+0000 7f8c5ca00640 1 Processor -- start 2024-03-28T01:06:19.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.452+0000 7f8c5ca00640 1 -- start start 2024-03-28T01:06:19.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c5ca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c580b51d0 0x7f8c580b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c5ca00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c580b8d00 0x7f8c5814ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c5ca00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f8c580b42f0 0x7f8c5814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c5ca00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5814a820 con 0x7f8c580b8d00 2024-03-28T01:06:19.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c5ca00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c580b55b0 con 0x7f8c580b42f0 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c5ca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580b9190 con 0x7f8c580b51d0 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c56a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c580b8d00 0x7f8c5814ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c56a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c580b8d00 0x7f8c5814ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58772/0 (socket says 172.21.15.33:58772) 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.453+0000 7f8c56a00640 1 -- 172.21.15.33:0/3784366743 learned_addr learned my addr 172.21.15.33:0/3784366743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c57400640 1 --2- 172.21.15.33:0/3784366743 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c580b51d0 0x7f8c580b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c57e00640 1 --2- 172.21.15.33:0/3784366743 >> v2:172.21.15.194:3300/0 conn(0x7f8c580b42f0 0x7f8c5814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c56a00640 1 -- 172.21.15.33:0/3784366743 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c580b51d0 msgr2=0x7f8c580b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c56a00640 1 --2- 172.21.15.33:0/3784366743 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c580b51d0 0x7f8c580b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c56a00640 1 -- 172.21.15.33:0/3784366743 >> v2:172.21.15.194:3300/0 conn(0x7f8c580b42f0 msgr2=0x7f8c5814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c56a00640 1 --2- 172.21.15.33:0/3784366743 >> v2:172.21.15.194:3300/0 conn(0x7f8c580b42f0 0x7f8c5814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c56a00640 1 -- 172.21.15.33:0/3784366743 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580b9800 con 0x7f8c580b8d00 2024-03-28T01:06:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.454+0000 7f8c57e00640 1 --2- 172.21.15.33:0/3784366743 >> v2:172.21.15.194:3300/0 conn(0x7f8c580b42f0 0x7f8c5814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.455+0000 7f8c56a00640 1 --2- 172.21.15.33:0/3784366743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b8d00 0x7f8c5814ac70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c580b4860 tx=0x7f8c4402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e755ef5ddbd5bc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.455+0000 7f8c56000640 1 -- 172.21.15.33:0/3784366743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44035020 con 0x7f8c580b8d00 2024-03-28T01:06:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.455+0000 7f8c56000640 1 -- 172.21.15.33:0/3784366743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4402d040 con 0x7f8c580b8d00 2024-03-28T01:06:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c56000640 1 -- 172.21.15.33:0/3784366743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44031a80 con 0x7f8c580b8d00 2024-03-28T01:06:19.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3784366743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b8d00 msgr2=0x7f8c5814ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3784366743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b8d00 0x7f8c5814ac70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c580b4860 tx=0x7f8c4402b900 comp rx=0 tx=0).stop 2024-03-28T01:06:19.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3784366743 shutdown_connections 2024-03-28T01:06:19.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3784366743 >> v2:172.21.15.194:3300/0 conn(0x7f8c580b42f0 0x7f8c5814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3784366743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b8d00 0x7f8c5814ac70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3784366743 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c580b51d0 0x7f8c580b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3784366743 >> 172.21.15.33:0/3784366743 conn(0x7f8c58072940 msgr2=0x7f8c58072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:19.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3784366743 shutdown_connections 2024-03-28T01:06:19.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.456+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3784366743 wait complete. 2024-03-28T01:06:19.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.458+0000 7f8c5ca00640 1 Processor -- start 2024-03-28T01:06:19.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.459+0000 7f8c5ca00640 1 -- start start 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.459+0000 7f8c5ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c580b42f0 0x7f8c580bc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.459+0000 7f8c5ca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b51d0 0x7f8c581e0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c5ca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c58143770 0x7f8c58141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c5ca00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580b39c0 con 0x7f8c580b51d0 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c5ca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5814b6f0 con 0x7f8c580b42f0 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c5ca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580b6b80 con 0x7f8c58143770 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b51d0 0x7f8c581e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b51d0 0x7f8c581e0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58786/0 (socket says 172.21.15.33:58786) 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 -- 172.21.15.33:0/3780560535 learned_addr learned my addr 172.21.15.33:0/3780560535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c57400640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c580b42f0 0x7f8c580bc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c57e00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c58143770 0x7f8c58141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 -- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c58143770 msgr2=0x7f8c58141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c58143770 0x7f8c58141c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 -- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c580b42f0 msgr2=0x7f8c580bc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c580b42f0 0x7f8c580bc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.460+0000 7f8c56a00640 1 -- 172.21.15.33:0/3780560535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c581d14d0 con 0x7f8c580b51d0 2024-03-28T01:06:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.461+0000 7f8c56a00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b51d0 0x7f8c581e0400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c4402b940 tx=0x7f8c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.461+0000 7f8c4f400640 1 -- 172.21.15.33:0/3780560535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44040070 con 0x7f8c580b51d0 2024-03-28T01:06:19.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.461+0000 7f8c4f400640 1 -- 172.21.15.33:0/3780560535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44039b70 con 0x7f8c580b51d0 2024-03-28T01:06:19.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.461+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c581441b0 con 0x7f8c580b51d0 2024-03-28T01:06:19.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.461+0000 7f8c4f400640 1 -- 172.21.15.33:0/3780560535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44035040 con 0x7f8c580b51d0 2024-03-28T01:06:19.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.461+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c58144e10 con 0x7f8c580b51d0 2024-03-28T01:06:19.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.462+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c58075310 con 0x7f8c580b51d0 2024-03-28T01:06:19.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.463+0000 7f8c4f400640 1 -- 172.21.15.33:0/3780560535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8c4404b060 con 0x7f8c580b51d0 2024-03-28T01:06:19.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.464+0000 7f8c4f400640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8c3806bc20 0x7f8c3806e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.464+0000 7f8c4f400640 1 -- 172.21.15.33:0/3780560535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8c440ac250 con 0x7f8c580b51d0 2024-03-28T01:06:19.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.464+0000 7f8c57400640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8c3806bc20 0x7f8c3806e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.465+0000 7f8c57400640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8c3806bc20 0x7f8c3806e0e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006fd0 tx=0x7f8c48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.469+0000 7f8c4f400640 1 -- 172.21.15.33:0/3780560535 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8c4407b100 con 0x7f8c580b51d0 2024-03-28T01:06:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.671+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8c5814a5e0 con 0x7f8c580b51d0 2024-03-28T01:06:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.672+0000 7f8c4f400640 1 -- 172.21.15.33:0/3780560535 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8c4407aa00 con 0x7f8c580b51d0 2024-03-28T01:06:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:19.675 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-03-28T01:06:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-03-28T01:06:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-03-28T01:06:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-03-28T01:06:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-03-28T01:06:19.675 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.674+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8c3806bc20 msgr2=0x7f8c3806e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.674+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8c3806bc20 0x7f8c3806e0e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006fd0 tx=0x7f8c48008040 comp rx=0 tx=0).stop 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.674+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b51d0 msgr2=0x7f8c581e0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.674+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b51d0 0x7f8c581e0400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c4402b940 tx=0x7f8c440046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 shutdown_connections 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8c3806bc20 0x7f8c3806e0e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c58143770 0x7f8c58141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580b51d0 0x7f8c581e0400 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 --2- 172.21.15.33:0/3780560535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c580b42f0 0x7f8c580bc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 >> 172.21.15.33:0/3780560535 conn(0x7f8c58072940 msgr2=0x7f8c581489c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 shutdown_connections 2024-03-28T01:06:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.675+0000 7f8c5ca00640 1 -- 172.21.15.33:0/3780560535 wait complete. 2024-03-28T01:06:19.695 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T01:06:19.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T01:06:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.904+0000 7f320a800640 1 Processor -- start 2024-03-28T01:06:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.904+0000 7f320a800640 1 -- start start 2024-03-28T01:06:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.905+0000 7f320a800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f32040727f0 0x7f320409f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.905+0000 7f320a800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3204140ed0 0x7f3204141320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f320a800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3204072bd0 con 0x7f32040a0d20 2024-03-28T01:06:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f320a800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32040b0920 con 0x7f3204140ed0 2024-03-28T01:06:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f320a800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32040b6c80 con 0x7f32040727f0 2024-03-28T01:06:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f3209e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3204140ed0 0x7f3204141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f3209e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3204140ed0 0x7f3204141320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58800/0 (socket says 172.21.15.33:58800) 2024-03-28T01:06:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f3209e00640 1 -- 172.21.15.33:0/3767010050 learned_addr learned my addr 172.21.15.33:0/3767010050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f3209400640 1 --2- 172.21.15.33:0/3767010050 >> v2:172.21.15.194:3300/0 conn(0x7f32040727f0 0x7f320409f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f3209e00640 1 -- 172.21.15.33:0/3767010050 >> v2:172.21.15.194:3300/0 conn(0x7f32040727f0 msgr2=0x7f320409f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.906+0000 7f3209e00640 1 --2- 172.21.15.33:0/3767010050 >> v2:172.21.15.194:3300/0 conn(0x7f32040727f0 0x7f320409f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.907+0000 7f3209e00640 1 -- 172.21.15.33:0/3767010050 >> v1:172.21.15.194:6789/0 conn(0x7f32040a0d20 legacy=0x7f32040b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.907+0000 7f3209e00640 1 -- 172.21.15.33:0/3767010050 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040b6f60 con 0x7f3204140ed0 2024-03-28T01:06:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.908+0000 7f3209e00640 1 --2- 172.21.15.33:0/3767010050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3204140ed0 0x7f3204141320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32040b1930 tx=0x7f31fc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd7f91070e3cb42 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.908+0000 7f3203e00640 1 -- 172.21.15.33:0/3767010050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc02c9f0 con 0x7f3204140ed0 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.908+0000 7f3203e00640 1 -- 172.21.15.33:0/3767010050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc02cb90 con 0x7f3204140ed0 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.908+0000 7f3203e00640 1 -- 172.21.15.33:0/3767010050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0343f0 con 0x7f3204140ed0 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 -- 172.21.15.33:0/3767010050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3204140ed0 msgr2=0x7f3204141320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 --2- 172.21.15.33:0/3767010050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3204140ed0 0x7f3204141320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32040b1930 tx=0x7f31fc02be50 comp rx=0 tx=0).stop 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 -- 172.21.15.33:0/3767010050 shutdown_connections 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 --2- 172.21.15.33:0/3767010050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3204140ed0 0x7f3204141320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 --2- 172.21.15.33:0/3767010050 >> v2:172.21.15.194:3300/0 conn(0x7f32040727f0 0x7f320409f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 -- 172.21.15.33:0/3767010050 >> 172.21.15.33:0/3767010050 conn(0x7f32040baa50 msgr2=0x7f32040bae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 -- 172.21.15.33:0/3767010050 shutdown_connections 2024-03-28T01:06:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.909+0000 7f320a800640 1 -- 172.21.15.33:0/3767010050 wait complete. 2024-03-28T01:06:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.910+0000 7f320a800640 1 Processor -- start 2024-03-28T01:06:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.911+0000 7f320a800640 1 -- start start 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.911+0000 7f320a800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32040727f0 0x7f320406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.911+0000 7f320a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32040a0d20 0x7f32041dca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.911+0000 7f320a800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3204140ed0 0x7f320413ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.911+0000 7f320a800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204073050 con 0x7f32040a0d20 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.911+0000 7f320a800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204072ed0 con 0x7f32040727f0 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.911+0000 7f320a800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3204141860 con 0x7f3204140ed0 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32040a0d20 0x7f32041dca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32040a0d20 0x7f32041dca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58810/0 (socket says 172.21.15.33:58810) 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 -- 172.21.15.33:0/3215503828 learned_addr learned my addr 172.21.15.33:0/3215503828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3209400640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32040727f0 0x7f320406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3209e00640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3204140ed0 0x7f320413ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 -- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3204140ed0 msgr2=0x7f320413ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3204140ed0 0x7f320413ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 -- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32040727f0 msgr2=0x7f320406f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32040727f0 0x7f320406f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.912+0000 7f3208a00640 1 -- 172.21.15.33:0/3215503828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320406ef10 con 0x7f32040a0d20 2024-03-28T01:06:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.913+0000 7f3208a00640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32040a0d20 0x7f32041dca50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31f000c430 tx=0x7f31f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.913+0000 7f3201600640 1 -- 172.21.15.33:0/3215503828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0011790 con 0x7f32040a0d20 2024-03-28T01:06:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.913+0000 7f3201600640 1 -- 172.21.15.33:0/3215503828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0017070 con 0x7f32040a0d20 2024-03-28T01:06:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.913+0000 7f3201600640 1 -- 172.21.15.33:0/3215503828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0011b70 con 0x7f32040a0d20 2024-03-28T01:06:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.913+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320406cc00 con 0x7f32040a0d20 2024-03-28T01:06:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.913+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320406d340 con 0x7f32040a0d20 2024-03-28T01:06:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.914+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3204075310 con 0x7f32040a0d20 2024-03-28T01:06:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.915+0000 7f3201600640 1 -- 172.21.15.33:0/3215503828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f31f0011d10 con 0x7f32040a0d20 2024-03-28T01:06:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.916+0000 7f3201600640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f31e406bbd0 0x7f31e406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.916+0000 7f3201600640 1 -- 172.21.15.33:0/3215503828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f31f008a580 con 0x7f32040a0d20 2024-03-28T01:06:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.916+0000 7f3209400640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f31e406bbd0 0x7f31e406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.917+0000 7f3209400640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f31e406bbd0 0x7f31e406e090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31f8006fd0 tx=0x7f31f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:19.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:19.921+0000 7f3201600640 1 -- 172.21.15.33:0/3215503828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f320406ef10 con 0x7f32040a0d20 2024-03-28T01:06:20.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.122+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3204147710 con 0x7f32040a0d20 2024-03-28T01:06:20.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.671+0000 7f3201600640 1 -- 172.21.15.33:0/3215503828 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f31f00593b0 con 0x7f32040a0d20 2024-03-28T01:06:20.674 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-03-28T01:06:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.675+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f31e406bbd0 msgr2=0x7f31e406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.675+0000 7f320a800640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f31e406bbd0 0x7f31e406e090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31f8006fd0 tx=0x7f31f8008040 comp rx=0 tx=0).stop 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.675+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32040a0d20 msgr2=0x7f32041dca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.675+0000 7f320a800640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32040a0d20 0x7f32041dca50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31f000c430 tx=0x7f31f000c8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 shutdown_connections 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f31e406bbd0 0x7f31e406e090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3204140ed0 0x7f320413ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32040a0d20 0x7f32041dca50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 --2- 172.21.15.33:0/3215503828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32040727f0 0x7f320406f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 >> 172.21.15.33:0/3215503828 conn(0x7f32040baa50 msgr2=0x7f32040b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 shutdown_connections 2024-03-28T01:06:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.676+0000 7f320a800640 1 -- 172.21.15.33:0/3215503828 wait complete. 2024-03-28T01:06:20.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T01:06:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.921+0000 7fbd67000640 1 Processor -- start 2024-03-28T01:06:20.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.921+0000 7fbd67000640 1 -- start start 2024-03-28T01:06:20.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.923+0000 7fbd67000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fbd600b1f90 0x7fbd600b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:20.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.923+0000 7fbd67000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd60071270 con 0x7fbd60145a90 2024-03-28T01:06:20.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.923+0000 7fbd67000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6006fc50 con 0x7fbd6013ed40 2024-03-28T01:06:20.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.923+0000 7fbd67000640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600b2370 con 0x7fbd600b1f90 2024-03-28T01:06:20.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.923+0000 7fbd66600640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbd60145a90 0x7fbd6014f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56426/0 (socket says 172.21.15.33:56426) 2024-03-28T01:06:20.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.923+0000 7fbd66600640 1 -- 172.21.15.33:0/3701658466 learned_addr learned my addr 172.21.15.33:0/3701658466 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:20.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.923+0000 7fbd65c00640 1 --2- 172.21.15.33:0/3701658466 >> v2:172.21.15.194:3300/0 conn(0x7fbd600b1f90 0x7fbd600b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:20.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.924+0000 7fbd5fe00640 1 -- 172.21.15.33:0/3701658466 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908363385 0 0) 0x7fbd60071270 con 0x7fbd60145a90 2024-03-28T01:06:20.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.924+0000 7fbd5fe00640 1 -- 172.21.15.33:0/3701658466 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd48003520 con 0x7fbd60145a90 2024-03-28T01:06:20.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.924+0000 7fbd65c00640 1 -- 172.21.15.33:0/3701658466 >> v1:172.21.15.194:6789/0 conn(0x7fbd6013ed40 legacy=0x7fbd60145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:20.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.924+0000 7fbd65c00640 1 -- 172.21.15.33:0/3701658466 >> v1:172.21.15.33:6789/0 conn(0x7fbd60145a90 legacy=0x7fbd6014f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:20.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.924+0000 7fbd65c00640 1 -- 172.21.15.33:0/3701658466 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6006fef0 con 0x7fbd600b1f90 2024-03-28T01:06:20.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.926+0000 7fbd65c00640 1 --2- 172.21.15.33:0/3701658466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 0x7fbd600b41b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009d90 tx=0x7fbd5402bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804b7b8d360d2ce6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.926+0000 7fbd5fe00640 1 -- 172.21.15.33:0/3701658466 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54039070 con 0x7fbd600b1f90 2024-03-28T01:06:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.926+0000 7fbd5fe00640 1 -- 172.21.15.33:0/3701658466 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5402cc70 con 0x7fbd600b1f90 2024-03-28T01:06:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.926+0000 7fbd5fe00640 1 -- 172.21.15.33:0/3701658466 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54034660 con 0x7fbd600b1f90 2024-03-28T01:06:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.927+0000 7fbd67000640 1 -- 172.21.15.33:0/3701658466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 msgr2=0x7fbd600b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.927+0000 7fbd67000640 1 --2- 172.21.15.33:0/3701658466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 0x7fbd600b41b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009d90 tx=0x7fbd5402bec0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.927+0000 7fbd67000640 1 -- 172.21.15.33:0/3701658466 shutdown_connections 2024-03-28T01:06:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.927+0000 7fbd67000640 1 --2- 172.21.15.33:0/3701658466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 0x7fbd600b41b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.927+0000 7fbd67000640 1 -- 172.21.15.33:0/3701658466 >> 172.21.15.33:0/3701658466 conn(0x7fbd600b8610 msgr2=0x7fbd600b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.927+0000 7fbd67000640 1 -- 172.21.15.33:0/3701658466 shutdown_connections 2024-03-28T01:06:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.927+0000 7fbd67000640 1 -- 172.21.15.33:0/3701658466 wait complete. 2024-03-28T01:06:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.929+0000 7fbd67000640 1 Processor -- start 2024-03-28T01:06:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.929+0000 7fbd67000640 1 -- start start 2024-03-28T01:06:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd67000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 0x7fbd601ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd67000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6013ed40 0x7fbd6013a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd67000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd60145a90 0x7fbd6013ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd67000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6013f320 con 0x7fbd6013ed40 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd67000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6013f1a0 con 0x7fbd600b1f90 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd67000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6013f4a0 con 0x7fbd60145a90 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd65200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6013ed40 0x7fbd6013a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd65200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6013ed40 0x7fbd6013a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58814/0 (socket says 172.21.15.33:58814) 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd65200640 1 -- 172.21.15.33:0/1173049863 learned_addr learned my addr 172.21.15.33:0/1173049863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.930+0000 7fbd65c00640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 0x7fbd601ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd66600640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd60145a90 0x7fbd6013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd65200640 1 -- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd60145a90 msgr2=0x7fbd6013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd65200640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd60145a90 0x7fbd6013ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd65200640 1 -- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 msgr2=0x7fbd601ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd65200640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 0x7fbd601ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd65200640 1 -- 172.21.15.33:0/1173049863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd601d1200 con 0x7fbd6013ed40 2024-03-28T01:06:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd65200640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6013ed40 0x7fbd6013a940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c400 tx=0x7fbd4c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd5d600640 1 -- 172.21.15.33:0/1173049863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0052d0 con 0x7fbd6013ed40 2024-03-28T01:06:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd5d600640 1 -- 172.21.15.33:0/1173049863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c005490 con 0x7fbd6013ed40 2024-03-28T01:06:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.931+0000 7fbd5d600640 1 -- 172.21.15.33:0/1173049863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c007260 con 0x7fbd6013ed40 2024-03-28T01:06:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.932+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd601cfb90 con 0x7fbd6013ed40 2024-03-28T01:06:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.932+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd601d0180 con 0x7fbd6013ed40 2024-03-28T01:06:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.933+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd60075310 con 0x7fbd6013ed40 2024-03-28T01:06:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.934+0000 7fbd5d600640 1 -- 172.21.15.33:0/1173049863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbd4c007400 con 0x7fbd6013ed40 2024-03-28T01:06:20.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.935+0000 7fbd5d600640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbd4006bc50 0x7fbd4006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:20.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.935+0000 7fbd5d600640 1 -- 172.21.15.33:0/1173049863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbd4c089d40 con 0x7fbd6013ed40 2024-03-28T01:06:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.935+0000 7fbd65c00640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbd4006bc50 0x7fbd4006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.936+0000 7fbd65c00640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbd4006bc50 0x7fbd4006e110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbd54036000 tx=0x7fbd54030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:20.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:20.940+0000 7fbd5d600640 1 -- 172.21.15.33:0/1173049863 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbd4c058a80 con 0x7fbd6013ed40 2024-03-28T01:06:21.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.143+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbd60070f20 con 0x7fbd6013ed40 2024-03-28T01:06:21.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.689+0000 7fbd5d600640 1 -- 172.21.15.33:0/1173049863 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbd4c058380 con 0x7fbd6013ed40 2024-03-28T01:06:21.692 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-03-28T01:06:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.693+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbd4006bc50 msgr2=0x7fbd4006e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.693+0000 7fbd67000640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbd4006bc50 0x7fbd4006e110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbd54036000 tx=0x7fbd54030000 comp rx=0 tx=0).stop 2024-03-28T01:06:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.693+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6013ed40 msgr2=0x7fbd6013a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.693+0000 7fbd67000640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6013ed40 0x7fbd6013a940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c400 tx=0x7fbd4c00c8c0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 shutdown_connections 2024-03-28T01:06:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbd4006bc50 0x7fbd4006e110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd60145a90 0x7fbd6013ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6013ed40 0x7fbd6013a940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 --2- 172.21.15.33:0/1173049863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd600b1f90 0x7fbd601ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 >> 172.21.15.33:0/1173049863 conn(0x7fbd600b8610 msgr2=0x7fbd60073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 shutdown_connections 2024-03-28T01:06:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.694+0000 7fbd67000640 1 -- 172.21.15.33:0/1173049863 wait complete. 2024-03-28T01:06:21.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T01:06:21.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.913+0000 7fb7b2400640 1 Processor -- start 2024-03-28T01:06:21.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.913+0000 7fb7b2400640 1 -- start start 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.914+0000 7fb7b2400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb7ac06d370 0x7fb7ac0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.914+0000 7fb7b2400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.914+0000 7fb7b2400640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ac0b2570 con 0x7fb7ac0b4aa0 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.914+0000 7fb7b2400640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ac070590 con 0x7fb7ac06d370 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.914+0000 7fb7b2400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac06d750 con 0x7fb7ac1459c0 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7ab400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb7ac06d370 0x7fb7ac0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7ab400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb7ac06d370 0x7fb7ac0b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58820/0 (socket says 172.21.15.33:58820) 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7ab400640 1 -- 172.21.15.33:0/3423579752 learned_addr learned my addr 172.21.15.33:0/3423579752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7abe00640 1 --2- 172.21.15.33:0/3423579752 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7ab400640 1 -- 172.21.15.33:0/3423579752 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 msgr2=0x7fb7ac14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7ab400640 1 --2- 172.21.15.33:0/3423579752 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7ab400640 1 -- 172.21.15.33:0/3423579752 >> v1:172.21.15.194:6789/0 conn(0x7fb7ac0b4aa0 legacy=0x7fb7ac1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.915+0000 7fb7ab400640 1 -- 172.21.15.33:0/3423579752 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac070870 con 0x7fb7ac06d370 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.916+0000 7fb7ab400640 1 --2- 172.21.15.33:0/3423579752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b4560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb794009870 tx=0x7fb79402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38716b565ab707b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7aa000640 1 -- 172.21.15.33:0/3423579752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794035020 con 0x7fb7ac06d370 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7aa000640 1 -- 172.21.15.33:0/3423579752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794031640 con 0x7fb7ac06d370 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7aa000640 1 -- 172.21.15.33:0/3423579752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79402d040 con 0x7fb7ac06d370 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 -- 172.21.15.33:0/3423579752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 msgr2=0x7fb7ac0b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 --2- 172.21.15.33:0/3423579752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b4560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb794009870 tx=0x7fb79402b8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 -- 172.21.15.33:0/3423579752 shutdown_connections 2024-03-28T01:06:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 --2- 172.21.15.33:0/3423579752 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 --2- 172.21.15.33:0/3423579752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b4560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 -- 172.21.15.33:0/3423579752 >> 172.21.15.33:0/3423579752 conn(0x7fb7ac0b5320 msgr2=0x7fb7ac0b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 -- 172.21.15.33:0/3423579752 shutdown_connections 2024-03-28T01:06:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.917+0000 7fb7b2400640 1 -- 172.21.15.33:0/3423579752 wait complete. 2024-03-28T01:06:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.919+0000 7fb7b2400640 1 Processor -- start 2024-03-28T01:06:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.919+0000 7fb7b2400640 1 -- start start 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7b2400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7b2400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7ac0b4aa0 0x7fb7ac1e0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7b2400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac1e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7b2400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0afa40 con 0x7fb7ac06d370 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7b2400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0af8c0 con 0x7fb7ac0b4aa0 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7b2400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac06f330 con 0x7fb7ac1459c0 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7ab400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7ab400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58824/0 (socket says 172.21.15.33:58824) 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7ab400640 1 -- 172.21.15.33:0/2654874168 learned_addr learned my addr 172.21.15.33:0/2654874168 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7aaa00640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7ac0b4aa0 0x7fb7ac1e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7abe00640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac1e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7ab400640 1 -- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 msgr2=0x7fb7ac1e0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7ab400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac1e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7ab400640 1 -- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7ac0b4aa0 msgr2=0x7fb7ac1e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.920+0000 7fb7ab400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7ac0b4aa0 0x7fb7ac1e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7ab400640 1 -- 172.21.15.33:0/2654874168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac1cb6a0 con 0x7fb7ac06d370 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7aaa00640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7ac0b4aa0 0x7fb7ac1e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7ab400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b10a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb794000c00 tx=0x7fb7940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7a3400640 1 -- 172.21.15.33:0/2654874168 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794040070 con 0x7fb7ac06d370 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7a3400640 1 -- 172.21.15.33:0/2654874168 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940399f0 con 0x7fb7ac06d370 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7a3400640 1 -- 172.21.15.33:0/2654874168 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794035040 con 0x7fb7ac06d370 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac1e11a0 con 0x7fb7ac06d370 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.921+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac1418a0 con 0x7fb7ac06d370 2024-03-28T01:06:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.922+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ac075310 con 0x7fb7ac06d370 2024-03-28T01:06:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.924+0000 7fb7a3400640 1 -- 172.21.15.33:0/2654874168 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb79404b060 con 0x7fb7ac06d370 2024-03-28T01:06:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.924+0000 7fb7a3400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb78806bbd0 0x7fb78806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.925+0000 7fb7a3400640 1 -- 172.21.15.33:0/2654874168 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb7940ac780 con 0x7fb7ac06d370 2024-03-28T01:06:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.925+0000 7fb7aaa00640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb78806bbd0 0x7fb78806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.925+0000 7fb7aaa00640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb78806bbd0 0x7fb78806e090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac1cb220 tx=0x7fb79c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:21.929+0000 7fb7a3400640 1 -- 172.21.15.33:0/2654874168 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb79407b340 con 0x7fb7ac06d370 2024-03-28T01:06:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.171+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7ac06fcf0 con 0x7fb7ac06d370 2024-03-28T01:06:22.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.694+0000 7fb7a3400640 1 -- 172.21.15.33:0/2654874168 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb79407ac40 con 0x7fb7ac06d370 2024-03-28T01:06:22.697 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-03-28T01:06:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb78806bbd0 msgr2=0x7fb78806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb78806bbd0 0x7fb78806e090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac1cb220 tx=0x7fb79c009040 comp rx=0 tx=0).stop 2024-03-28T01:06:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 msgr2=0x7fb7ac0b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b10a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb794000c00 tx=0x7fb7940046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 shutdown_connections 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb78806bbd0 0x7fb78806e090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7ac1459c0 0x7fb7ac1e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7ac0b4aa0 0x7fb7ac1e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 --2- 172.21.15.33:0/2654874168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ac06d370 0x7fb7ac0b10a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 >> 172.21.15.33:0/2654874168 conn(0x7fb7ac0b5320 msgr2=0x7fb7ac0b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 shutdown_connections 2024-03-28T01:06:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.698+0000 7fb7b2400640 1 -- 172.21.15.33:0/2654874168 wait complete. 2024-03-28T01:06:22.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:22.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.919+0000 7f4021600640 1 Processor -- start 2024-03-28T01:06:22.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.919+0000 7f4021600640 1 -- start start 2024-03-28T01:06:22.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.920+0000 7f4021600640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f401c0b2130 0x7f401c13edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.920+0000 7f4021600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c148cf0 0x7f401c14f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.920+0000 7f4021600640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c0b2510 con 0x7f401c13f330 2024-03-28T01:06:22.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.920+0000 7f4021600640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c0b4670 con 0x7f401c0b2130 2024-03-28T01:06:22.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.920+0000 7f4021600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c06bef0 con 0x7f401c148cf0 2024-03-28T01:06:22.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.920+0000 7f4013400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f401c13f330 0x7f401c1485e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40966/0 (socket says 172.21.15.33:40966) 2024-03-28T01:06:22.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f4013400640 1 -- 172.21.15.33:0/1181174349 learned_addr learned my addr 172.21.15.33:0/1181174349 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:22.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401be00640 1 --2- 172.21.15.33:0/1181174349 >> v2:172.21.15.194:3300/0 conn(0x7f401c0b2130 0x7f401c13edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:22.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f4020c00640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c148cf0 0x7f401c14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:22.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124871981 0 0) 0x7f401c0b2510 con 0x7f401c13f330 2024-03-28T01:06:22.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4000003520 con 0x7f401c13f330 2024-03-28T01:06:22.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4051571529 0 0) 0x7f4000003520 con 0x7f401c13f330 2024-03-28T01:06:22.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c148cf0 msgr2=0x7f401c14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:22.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c148cf0 0x7f401c14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 >> v2:172.21.15.194:3300/0 conn(0x7f401c0b2130 msgr2=0x7f401c13edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 --2- 172.21.15.33:0/1181174349 >> v2:172.21.15.194:3300/0 conn(0x7f401c0b2130 0x7f401c13edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0b4c40 con 0x7f401c13f330 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f4004001670 con 0x7f401c13f330 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 >> v1:172.21.15.33:6789/0 conn(0x7f401c13f330 legacy=0x7f401c1485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.921+0000 7f401b400640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f4000003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401b400640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c148cf0 0x7f4000004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401b400640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4000004d40 0x7f40000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0b4670 con 0x7f40000039b0 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c06bef0 con 0x7f4000004d40 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400000ba90 con 0x7f401c148cf0 2024-03-28T01:06:22.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401be00640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f4000003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:22.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401be00640 1 -- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c148cf0 msgr2=0x7f4000004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:22.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401be00640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c148cf0 0x7f4000004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401be00640 1 -- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4000004d40 msgr2=0x7f40000051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-28T01:06:22.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401be00640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4000004d40 0x7f40000051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.922+0000 7f401be00640 1 -- 172.21.15.33:0/1181174349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000003520 con 0x7f40000039b0 2024-03-28T01:06:22.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.923+0000 7f401be00640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f4000003dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f400c009ac0 tx=0x7f4000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768bab44bd0592ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.923+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c039070 con 0x7f40000039b0 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.923+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c02c710 con 0x7f40000039b0 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.923+0000 7f401b400640 1 -- 172.21.15.33:0/1181174349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c009690 con 0x7f40000039b0 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.924+0000 7f4021600640 1 -- 172.21.15.33:0/1181174349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 msgr2=0x7f4000003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.924+0000 7f4021600640 1 --2- 172.21.15.33:0/1181174349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f4000003dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f400c009ac0 tx=0x7f4000005830 comp rx=0 tx=0).stop 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.924+0000 7f4020c00640 1 -- 172.21.15.33:0/1181174349 reap_dead start 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.924+0000 7f4021600640 1 -- 172.21.15.33:0/1181174349 shutdown_connections 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.924+0000 7f4021600640 1 -- 172.21.15.33:0/1181174349 >> 172.21.15.33:0/1181174349 conn(0x7f401c0b89c0 msgr2=0x7f401c0b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.924+0000 7f4021600640 1 -- 172.21.15.33:0/1181174349 shutdown_connections 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.924+0000 7f4021600640 1 -- 172.21.15.33:0/1181174349 wait complete. 2024-03-28T01:06:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.926+0000 7f4021600640 1 Processor -- start 2024-03-28T01:06:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.926+0000 7f4021600640 1 -- start start 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f4021600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f401c0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f4021600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f401c0b0790 0x7f401c1dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f4021600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c0afcb0 0x7f401c1dd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f4021600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c072d20 con 0x7f40000039b0 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f4021600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0b4ec0 con 0x7f401c0b0790 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f4021600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0b0c70 con 0x7f401c0afcb0 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f401be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f401c0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f401be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f401c0b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36148/0 (socket says 172.21.15.33:36148) 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.927+0000 7f401be00640 1 -- 172.21.15.33:0/4222337502 learned_addr learned my addr 172.21.15.33:0/4222337502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:22.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4013400640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f401c0b0790 0x7f401c1dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4020c00640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c0afcb0 0x7f401c1dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f401be00640 1 -- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c0afcb0 msgr2=0x7f401c1dd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f401be00640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c0afcb0 0x7f401c1dd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f401be00640 1 -- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f401c0b0790 msgr2=0x7f401c1dcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f401be00640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f401c0b0790 0x7f401c1dcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f401be00640 1 -- 172.21.15.33:0/4222337502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0add60 con 0x7f40000039b0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4013400640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f401c0b0790 0x7f401c1dcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f401be00640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f401c0b16e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f400c000c00 tx=0x7f400c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4018c00640 1 -- 172.21.15.33:0/4222337502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c043020 con 0x7f40000039b0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4018c00640 1 -- 172.21.15.33:0/4222337502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c0044a0 con 0x7f40000039b0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4018c00640 1 -- 172.21.15.33:0/4222337502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c039040 con 0x7f40000039b0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c0ae040 con 0x7f40000039b0 2024-03-28T01:06:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.928+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c0aea90 con 0x7f40000039b0 2024-03-28T01:06:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.930+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c0689d0 con 0x7f40000039b0 2024-03-28T01:06:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.931+0000 7f4018c00640 1 -- 172.21.15.33:0/4222337502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f400c046020 con 0x7f40000039b0 2024-03-28T01:06:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.932+0000 7f4018c00640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3ff806bbd0 0x7f3ff806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:22.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.932+0000 7f4018c00640 1 -- 172.21.15.33:0/4222337502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f400c0ac6e0 con 0x7f40000039b0 2024-03-28T01:06:22.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.932+0000 7f4013400640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3ff806bbd0 0x7f3ff806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:22.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.933+0000 7f4013400640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3ff806bbd0 0x7f3ff806e090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4004000c00 tx=0x7f4004009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:22.937+0000 7f4018c00640 1 -- 172.21.15.33:0/4222337502 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f401c0689d0 con 0x7f40000039b0 2024-03-28T01:06:23.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.139+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f401c075310 con 0x7f40000039b0 2024-03-28T01:06:23.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.141+0000 7f4018c00640 1 -- 172.21.15.33:0/4222337502 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16618 (secure 0 0 0) 0x7f400c04d030 con 0x7f40000039b0 2024-03-28T01:06:23.144 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:23.144 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:22.173045+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3ff806bbd0 msgr2=0x7f3ff806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3ff806bbd0 0x7f3ff806e090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4004000c00 tx=0x7f4004009040 comp rx=0 tx=0).stop 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 msgr2=0x7f401c0b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f401c0b16e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f400c000c00 tx=0x7f400c004730 comp rx=0 tx=0).stop 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 shutdown_connections 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3ff806bbd0 0x7f3ff806e090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f401c0afcb0 0x7f401c1dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f401c0b0790 0x7f401c1dcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 --2- 172.21.15.33:0/4222337502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40000039b0 0x7f401c0b16e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 >> 172.21.15.33:0/4222337502 conn(0x7f401c0b89c0 msgr2=0x7f401c0b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 shutdown_connections 2024-03-28T01:06:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.144+0000 7f4021600640 1 -- 172.21.15.33:0/4222337502 wait complete. 2024-03-28T01:06:23.165 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.371+0000 7f659be00640 1 Processor -- start 2024-03-28T01:06:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.372+0000 7f659be00640 1 -- start start 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f659be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f65940b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f659be00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f65940b4fa0 0x7f659413f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f659be00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659406d7c0 con 0x7f65940b2d00 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f659be00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65940b10f0 con 0x7f65940b4fa0 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f659be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65940b0aa0 con 0x7f659406d3e0 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f6599e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f65940b4fa0 0x7f659413f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f6599e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f65940b4fa0 0x7f659413f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36160/0 (socket says 172.21.15.33:36160) 2024-03-28T01:06:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.373+0000 7f6599e00640 1 -- 172.21.15.33:0/895835086 learned_addr learned my addr 172.21.15.33:0/895835086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.374+0000 7f6599400640 1 --2- 172.21.15.33:0/895835086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f65940b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.374+0000 7f6599e00640 1 -- 172.21.15.33:0/895835086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 msgr2=0x7f65940b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.374+0000 7f6599e00640 1 --2- 172.21.15.33:0/895835086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f65940b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.374+0000 7f6599e00640 1 -- 172.21.15.33:0/895835086 >> v1:172.21.15.194:6789/0 conn(0x7f65940b2d00 legacy=0x7f659413eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.374+0000 7f6599e00640 1 -- 172.21.15.33:0/895835086 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940b0d80 con 0x7f65940b4fa0 2024-03-28T01:06:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.374+0000 7f6599400640 1 --2- 172.21.15.33:0/895835086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f65940b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.375+0000 7f6599e00640 1 --2- 172.21.15.33:0/895835086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413f5b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f658c009e50 tx=0x7f658c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eff024133deef5a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f6593e00640 1 -- 172.21.15.33:0/895835086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c02ca70 con 0x7f65940b4fa0 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f6593e00640 1 -- 172.21.15.33:0/895835086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c02cc10 con 0x7f65940b4fa0 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f6593e00640 1 -- 172.21.15.33:0/895835086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0343f0 con 0x7f65940b4fa0 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 -- 172.21.15.33:0/895835086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 msgr2=0x7f659413f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 --2- 172.21.15.33:0/895835086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413f5b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f658c009e50 tx=0x7f658c02bed0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 -- 172.21.15.33:0/895835086 shutdown_connections 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 --2- 172.21.15.33:0/895835086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413f5b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 --2- 172.21.15.33:0/895835086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f65940b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 -- 172.21.15.33:0/895835086 >> 172.21.15.33:0/895835086 conn(0x7f659406b030 msgr2=0x7f659406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 -- 172.21.15.33:0/895835086 shutdown_connections 2024-03-28T01:06:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.376+0000 7f659be00640 1 -- 172.21.15.33:0/895835086 wait complete. 2024-03-28T01:06:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.378+0000 7f659be00640 1 Processor -- start 2024-03-28T01:06:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.378+0000 7f659be00640 1 -- start start 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f659be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f6594141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f659be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65940b2d00 0x7f6594145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f659be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f659be00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940b0150 con 0x7f65940b4fa0 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f659be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940affd0 con 0x7f65940b2d00 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f659be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659414fab0 con 0x7f659406d3e0 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f6599e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f6599e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36168/0 (socket says 172.21.15.33:36168) 2024-03-28T01:06:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f6599e00640 1 -- 172.21.15.33:0/2126432730 learned_addr learned my addr 172.21.15.33:0/2126432730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f6598a00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65940b2d00 0x7f6594145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f6599e00640 1 -- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 msgr2=0x7f6594141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.379+0000 7f6599400640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f6594141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6599e00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f6594141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6599e00640 1 -- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65940b2d00 msgr2=0x7f6594145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6599e00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65940b2d00 0x7f6594145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6599e00640 1 -- 172.21.15.33:0/2126432730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659413fbb0 con 0x7f65940b4fa0 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6598a00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65940b2d00 0x7f6594145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6599e00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413b1f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f658c009f70 tx=0x7f658c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6591600640 1 -- 172.21.15.33:0/2126432730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c043020 con 0x7f65940b4fa0 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6591600640 1 -- 172.21.15.33:0/2126432730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c03e450 con 0x7f65940b4fa0 2024-03-28T01:06:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65941420e0 con 0x7f65940b4fa0 2024-03-28T01:06:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f6591600640 1 -- 172.21.15.33:0/2126432730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c009510 con 0x7f65940b4fa0 2024-03-28T01:06:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.380+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6594143020 con 0x7f65940b4fa0 2024-03-28T01:06:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.381+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659413c5e0 con 0x7f65940b4fa0 2024-03-28T01:06:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.382+0000 7f6591600640 1 -- 172.21.15.33:0/2126432730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f658c046020 con 0x7f65940b4fa0 2024-03-28T01:06:23.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.383+0000 7f6591600640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f657406bc00 0x7f657406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.383+0000 7f6591600640 1 -- 172.21.15.33:0/2126432730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f658c0ad220 con 0x7f65940b4fa0 2024-03-28T01:06:23.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.384+0000 7f6599400640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f657406bc00 0x7f657406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.384+0000 7f6599400640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f657406bc00 0x7f657406e0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6588006fd0 tx=0x7f6588008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.388+0000 7f6591600640 1 -- 172.21.15.33:0/2126432730 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f658c04d030 con 0x7f65940b4fa0 2024-03-28T01:06:23.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.589+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6594075380 con 0x7f65940b4fa0 2024-03-28T01:06:23.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.591+0000 7f6591600640 1 -- 172.21.15.33:0/2126432730 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16618 (secure 0 0 0) 0x7f658c07bd00 con 0x7f65940b4fa0 2024-03-28T01:06:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:22.173045+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.593+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f657406bc00 msgr2=0x7f657406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.593+0000 7f659be00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f657406bc00 0x7f657406e0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6588006fd0 tx=0x7f6588008040 comp rx=0 tx=0).stop 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.593+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 msgr2=0x7f659413b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.593+0000 7f659be00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413b1f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f658c009f70 tx=0x7f658c004a40 comp rx=0 tx=0).stop 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 shutdown_connections 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f657406bc00 0x7f657406e0c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940b4fa0 0x7f659413b1f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65940b2d00 0x7f6594145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 --2- 172.21.15.33:0/2126432730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f659406d3e0 0x7f6594141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 >> 172.21.15.33:0/2126432730 conn(0x7f659406b030 msgr2=0x7f659413cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 shutdown_connections 2024-03-28T01:06:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.594+0000 7f659be00640 1 -- 172.21.15.33:0/2126432730 wait complete. 2024-03-28T01:06:23.615 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T01:06:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.835+0000 7fe38b200640 1 Processor -- start 2024-03-28T01:06:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.836+0000 7fe38b200640 1 -- start start 2024-03-28T01:06:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe38b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe384067040 0x7fe3840b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe38b200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe38409c360 0x7fe384145270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe38b200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fe384142380 0x7fe384142830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe38b200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe384142d70 con 0x7fe38409c360 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe38b200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe384067420 con 0x7fe384142380 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe38b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38409c8b0 con 0x7fe384067040 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe37be00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe38409c360 0x7fe384145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe37be00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe38409c360 0x7fe384145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36180/0 (socket says 172.21.15.33:36180) 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.837+0000 7fe37be00640 1 -- 172.21.15.33:0/3008968114 learned_addr learned my addr 172.21.15.33:0/3008968114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe383e00640 1 --2- 172.21.15.33:0/3008968114 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe384067040 0x7fe3840b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe389200640 1 --2- 172.21.15.33:0/3008968114 >> v2:172.21.15.194:3300/0 conn(0x7fe384142380 0x7fe384142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe37be00640 1 -- 172.21.15.33:0/3008968114 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe384067040 msgr2=0x7fe3840b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe37be00640 1 --2- 172.21.15.33:0/3008968114 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe384067040 0x7fe3840b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe37be00640 1 -- 172.21.15.33:0/3008968114 >> v2:172.21.15.194:3300/0 conn(0x7fe384142380 msgr2=0x7fe384142830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe37be00640 1 --2- 172.21.15.33:0/3008968114 >> v2:172.21.15.194:3300/0 conn(0x7fe384142380 0x7fe384142830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe37be00640 1 -- 172.21.15.33:0/3008968114 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38409cb90 con 0x7fe38409c360 2024-03-28T01:06:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.838+0000 7fe389200640 1 --2- 172.21.15.33:0/3008968114 >> v2:172.21.15.194:3300/0 conn(0x7fe384142380 0x7fe384142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.839+0000 7fe37be00640 1 --2- 172.21.15.33:0/3008968114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe38409c360 0x7fe384145270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3840b5a20 tx=0x7fe36c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf268725984ea650 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.839+0000 7fe383400640 1 -- 172.21.15.33:0/3008968114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c038070 con 0x7fe38409c360 2024-03-28T01:06:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.839+0000 7fe383400640 1 -- 172.21.15.33:0/3008968114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c031740 con 0x7fe38409c360 2024-03-28T01:06:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.839+0000 7fe383400640 1 -- 172.21.15.33:0/3008968114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c02f050 con 0x7fe38409c360 2024-03-28T01:06:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 -- 172.21.15.33:0/3008968114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe38409c360 msgr2=0x7fe384145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 --2- 172.21.15.33:0/3008968114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe38409c360 0x7fe384145270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3840b5a20 tx=0x7fe36c02b7e0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 -- 172.21.15.33:0/3008968114 shutdown_connections 2024-03-28T01:06:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 --2- 172.21.15.33:0/3008968114 >> v2:172.21.15.194:3300/0 conn(0x7fe384142380 0x7fe384142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 --2- 172.21.15.33:0/3008968114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe38409c360 0x7fe384145270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 --2- 172.21.15.33:0/3008968114 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe384067040 0x7fe3840b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 -- 172.21.15.33:0/3008968114 >> 172.21.15.33:0/3008968114 conn(0x7fe3840b5c40 msgr2=0x7fe3840b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.840+0000 7fe38b200640 1 -- 172.21.15.33:0/3008968114 shutdown_connections 2024-03-28T01:06:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.841+0000 7fe38b200640 1 -- 172.21.15.33:0/3008968114 wait complete. 2024-03-28T01:06:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.842+0000 7fe38b200640 1 Processor -- start 2024-03-28T01:06:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.843+0000 7fe38b200640 1 -- start start 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe38b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe384067040 0x7fe3841daa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe38b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38409c360 0x7fe3840b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe38b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe384142380 0x7fe384071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe38b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3841457b0 con 0x7fe384142380 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe38b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840b4b00 con 0x7fe384067040 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe38b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840b3b40 con 0x7fe38409c360 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe389200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe384142380 0x7fe384071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe389200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe384142380 0x7fe384071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36196/0 (socket says 172.21.15.33:36196) 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe389200640 1 -- 172.21.15.33:0/4218332664 learned_addr learned my addr 172.21.15.33:0/4218332664 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe383e00640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe384067040 0x7fe3841daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.844+0000 7fe37be00640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38409c360 0x7fe3840b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe389200640 1 -- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38409c360 msgr2=0x7fe3840b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe389200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38409c360 0x7fe3840b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe389200640 1 -- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe384067040 msgr2=0x7fe3841daa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe389200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe384067040 0x7fe3841daa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe389200640 1 -- 172.21.15.33:0/4218332664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3841402a0 con 0x7fe384142380 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe37be00640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38409c360 0x7fe3840b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe389200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe384142380 0x7fe384071f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00cd50 tx=0x7fe37c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe380c00640 1 -- 172.21.15.33:0/4218332664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00f680 con 0x7fe384142380 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe380c00640 1 -- 172.21.15.33:0/4218332664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c00f840 con 0x7fe384142380 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe380c00640 1 -- 172.21.15.33:0/4218332664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00fc30 con 0x7fe384142380 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3840b0d50 con 0x7fe384142380 2024-03-28T01:06:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.845+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3840b0500 con 0x7fe384142380 2024-03-28T01:06:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.847+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3840b4490 con 0x7fe384142380 2024-03-28T01:06:23.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.849+0000 7fe380c00640 1 -- 172.21.15.33:0/4218332664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe37c00e3e0 con 0x7fe384142380 2024-03-28T01:06:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.850+0000 7fe380c00640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe36006bbd0 0x7fe36006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.850+0000 7fe380c00640 1 -- 172.21.15.33:0/4218332664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe37c08b790 con 0x7fe384142380 2024-03-28T01:06:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.851+0000 7fe383e00640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe36006bbd0 0x7fe36006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.851+0000 7fe383e00640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe36006bbd0 0x7fe36006e090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe374006fd0 tx=0x7fe374008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:23.855+0000 7fe380c00640 1 -- 172.21.15.33:0/4218332664 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe3840b4490 con 0x7fe384142380 2024-03-28T01:06:24.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.094+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe384075310 con 0x7fe384142380 2024-03-28T01:06:24.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.728+0000 7fe380c00640 1 -- 172.21.15.33:0/4218332664 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe37c014030 con 0x7fe384142380 2024-03-28T01:06:24.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe36006bbd0 msgr2=0x7fe36006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:24.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe36006bbd0 0x7fe36006e090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe374006fd0 tx=0x7fe374008040 comp rx=0 tx=0).stop 2024-03-28T01:06:24.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe384142380 msgr2=0x7fe384071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:24.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe384142380 0x7fe384071f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00cd50 tx=0x7fe37c00a650 comp rx=0 tx=0).stop 2024-03-28T01:06:24.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 shutdown_connections 2024-03-28T01:06:24.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe36006bbd0 0x7fe36006e090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe384142380 0x7fe384071f10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38409c360 0x7fe3840b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 --2- 172.21.15.33:0/4218332664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe384067040 0x7fe3841daa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.732+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 >> 172.21.15.33:0/4218332664 conn(0x7fe3840b5c40 msgr2=0x7fe384143ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:24.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.733+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 shutdown_connections 2024-03-28T01:06:24.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.733+0000 7fe38b200640 1 -- 172.21.15.33:0/4218332664 wait complete. 2024-03-28T01:06:24.756 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T01:06:24.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T01:06:24.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.978+0000 7f5d25e00640 1 Processor -- start 2024-03-28T01:06:24.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.978+0000 7f5d25e00640 1 -- start start 2024-03-28T01:06:24.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.979+0000 7f5d25e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f5d200b7830 0x7f5d201452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.979+0000 7f5d25e00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2013fa70 con 0x7f5d200b1b20 2024-03-28T01:06:24.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.979+0000 7f5d25e00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d200b6490 con 0x7f5d20072d20 2024-03-28T01:06:24.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.979+0000 7f5d25e00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d200b5f60 con 0x7f5d200b7830 2024-03-28T01:06:24.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.979+0000 7f5d1f400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5d200b1b20 0x7f5d2014fb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40978/0 (socket says 172.21.15.33:40978) 2024-03-28T01:06:24.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1f400640 1 -- 172.21.15.33:0/4047043455 learned_addr learned my addr 172.21.15.33:0/4047043455 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/4047043455 >> v2:172.21.15.194:3300/0 conn(0x7f5d200b7830 0x7f5d201452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939254247 0 0) 0x7f5d2013fa70 con 0x7f5d200b1b20 2024-03-28T01:06:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d04003520 con 0x7f5d200b1b20 2024-03-28T01:06:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101795365 0 0) 0x7f5d04003520 con 0x7f5d200b1b20 2024-03-28T01:06:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 >> v2:172.21.15.194:3300/0 conn(0x7f5d200b7830 msgr2=0x7f5d201452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 --2- 172.21.15.33:0/4047043455 >> v2:172.21.15.194:3300/0 conn(0x7f5d200b7830 0x7f5d201452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 >> v1:172.21.15.194:6789/0 conn(0x7f5d20072d20 legacy=0x7f5d200b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200b7c10 con 0x7f5d200b1b20 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f5d180019f0 con 0x7f5d200b1b20 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.980+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 >> v1:172.21.15.33:6789/0 conn(0x7f5d200b1b20 legacy=0x7f5d2014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1e000640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04003a00 0x7f5d04003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1e000640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d04004800 0x7f5d04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1e000640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d040052c0 0x7f5d0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200b5f60 con 0x7f5d04003a00 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0400c200 con 0x7f5d040052c0 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0400c400 con 0x7f5d04004800 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04003a00 0x7f5d04003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1f400640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d040052c0 0x7f5d0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:24.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1ea00640 1 -- 172.21.15.33:0/4047043455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d04004800 msgr2=0x7f5d04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d04004800 0x7f5d04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1ea00640 1 -- 172.21.15.33:0/4047043455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d040052c0 msgr2=0x7f5d0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d040052c0 0x7f5d0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1ea00640 1 -- 172.21.15.33:0/4047043455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04003520 con 0x7f5d04003a00 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.981+0000 7f5d1f400640 1 -- 172.21.15.33:0/4047043455 reap_dead start 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.982+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04003a00 0x7f5d04003e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009ac0 tx=0x7f5d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc21991e332e26b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10039070 con 0x7f5d04003a00 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1002c6b0 con 0x7f5d04003a00 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d1e000640 1 -- 172.21.15.33:0/4047043455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10009690 con 0x7f5d04003a00 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d25e00640 1 -- 172.21.15.33:0/4047043455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04003a00 msgr2=0x7f5d04003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d25e00640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04003a00 0x7f5d04003e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009ac0 tx=0x7f5d04005830 comp rx=0 tx=0).stop 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d25e00640 1 -- 172.21.15.33:0/4047043455 shutdown_connections 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d25e00640 1 --2- 172.21.15.33:0/4047043455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04003a00 0x7f5d04003e40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.983+0000 7f5d25e00640 1 -- 172.21.15.33:0/4047043455 >> 172.21.15.33:0/4047043455 conn(0x7f5d2006b030 msgr2=0x7f5d2006b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:24.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.984+0000 7f5d25e00640 1 -- 172.21.15.33:0/4047043455 shutdown_connections 2024-03-28T01:06:24.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.984+0000 7f5d25e00640 1 -- 172.21.15.33:0/4047043455 wait complete. 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.986+0000 7f5d25e00640 1 Processor -- start 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.986+0000 7f5d25e00640 1 -- start start 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d25e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d04003a00 0x7f5d201e0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d25e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d201e0dd0 0x7f5d201dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d25e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d201dd0f0 0x7f5d201dd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d25e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200b6c50 con 0x7f5d201e0dd0 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d25e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200b5c10 con 0x7f5d04003a00 2024-03-28T01:06:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d25e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200b6dd0 con 0x7f5d201dd0f0 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d16000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d201e0dd0 0x7f5d201dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d16000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d201e0dd0 0x7f5d201dcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36206/0 (socket says 172.21.15.33:36206) 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d16000640 1 -- 172.21.15.33:0/373878942 learned_addr learned my addr 172.21.15.33:0/373878942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d04003a00 0x7f5d201e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d1f400640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d201dd0f0 0x7f5d201dd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d16000640 1 -- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d201dd0f0 msgr2=0x7f5d201dd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d16000640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d201dd0f0 0x7f5d201dd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d16000640 1 -- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d04003a00 msgr2=0x7f5d201e0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.988+0000 7f5d16000640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d04003a00 0x7f5d201e0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d16000640 1 -- 172.21.15.33:0/373878942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2006e0f0 con 0x7f5d201e0dd0 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d1f400640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d201dd0f0 0x7f5d201dd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d16000640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d201e0dd0 0x7f5d201dcbb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c460 tx=0x7f5d0800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d17e00640 1 -- 172.21.15.33:0/373878942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d080052d0 con 0x7f5d201e0dd0 2024-03-28T01:06:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d17e00640 1 -- 172.21.15.33:0/373878942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d08005490 con 0x7f5d201e0dd0 2024-03-28T01:06:24.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d17e00640 1 -- 172.21.15.33:0/373878942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08012870 con 0x7f5d201e0dd0 2024-03-28T01:06:24.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d201e08d0 con 0x7f5d201e0dd0 2024-03-28T01:06:24.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.989+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d201d2340 con 0x7f5d201e0dd0 2024-03-28T01:06:24.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.991+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d20075310 con 0x7f5d201e0dd0 2024-03-28T01:06:24.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.992+0000 7f5d17e00640 1 -- 172.21.15.33:0/373878942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5d08012a10 con 0x7f5d201e0dd0 2024-03-28T01:06:24.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.992+0000 7f5d17e00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5cfc06bbd0 0x7f5cfc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:24.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.993+0000 7f5d17e00640 1 -- 172.21.15.33:0/373878942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d0808a5b0 con 0x7f5d201e0dd0 2024-03-28T01:06:24.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.993+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5cfc06bbd0 0x7f5cfc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:24.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.993+0000 7f5d1ea00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5cfc06bbd0 0x7f5cfc06e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d10036000 tx=0x7f5d10030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:24.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:24.997+0000 7f5d17e00640 1 -- 172.21.15.33:0/373878942 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5d08059130 con 0x7f5d201e0dd0 2024-03-28T01:06:25.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.237+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5d20143ae0 con 0x7f5d201e0dd0 2024-03-28T01:06:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.731+0000 7f5d17e00640 1 -- 172.21.15.33:0/373878942 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5d08058a30 con 0x7f5d201e0dd0 2024-03-28T01:06:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.735+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5cfc06bbd0 msgr2=0x7f5cfc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.735+0000 7f5d25e00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5cfc06bbd0 0x7f5cfc06e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d10036000 tx=0x7f5d10030000 comp rx=0 tx=0).stop 2024-03-28T01:06:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.735+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d201e0dd0 msgr2=0x7f5d201dcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.735+0000 7f5d25e00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d201e0dd0 0x7f5d201dcbb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c460 tx=0x7f5d0800c920 comp rx=0 tx=0).stop 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 shutdown_connections 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5cfc06bbd0 0x7f5cfc06e090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d201dd0f0 0x7f5d201dd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d201e0dd0 0x7f5d201dcbb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 --2- 172.21.15.33:0/373878942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d04003a00 0x7f5d201e0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 >> 172.21.15.33:0/373878942 conn(0x7f5d2006b030 msgr2=0x7f5d201448a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 shutdown_connections 2024-03-28T01:06:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.736+0000 7f5d25e00640 1 -- 172.21.15.33:0/373878942 wait complete. 2024-03-28T01:06:25.764 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T01:06:25.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T01:06:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.977+0000 7f6b55400640 1 Processor -- start 2024-03-28T01:06:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.977+0000 7f6b55400640 1 -- start start 2024-03-28T01:06:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.979+0000 7f6b55400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b500b56e0 0x7f6b50141110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.979+0000 7f6b55400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f6b50141650 0x7f6b5014a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.979+0000 7f6b55400640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b500b5ac0 con 0x7f6b500b46b0 2024-03-28T01:06:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.979+0000 7f6b55400640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b500b6920 con 0x7f6b50141650 2024-03-28T01:06:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.979+0000 7f6b55400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b9d40 con 0x7f6b500b56e0 2024-03-28T01:06:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.979+0000 7f6b4f200640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6b500b46b0 0x7f6b500b75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40990/0 (socket says 172.21.15.33:40990) 2024-03-28T01:06:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.979+0000 7f6b4f200640 1 -- 172.21.15.33:0/3690175390 learned_addr learned my addr 172.21.15.33:0/3690175390 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4e800640 1 --2- 172.21.15.33:0/3690175390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b500b56e0 0x7f6b50141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4de00640 1 --2- 172.21.15.33:0/3690175390 >> v2:172.21.15.194:3300/0 conn(0x7f6b50141650 0x7f6b5014a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4d400640 1 -- 172.21.15.33:0/3690175390 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101281912 0 0) 0x7f6b500b5ac0 con 0x7f6b500b46b0 2024-03-28T01:06:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4d400640 1 -- 172.21.15.33:0/3690175390 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b34003520 con 0x7f6b500b46b0 2024-03-28T01:06:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4e800640 1 -- 172.21.15.33:0/3690175390 >> v2:172.21.15.194:3300/0 conn(0x7f6b50141650 msgr2=0x7f6b5014a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4e800640 1 --2- 172.21.15.33:0/3690175390 >> v2:172.21.15.194:3300/0 conn(0x7f6b50141650 0x7f6b5014a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4e800640 1 -- 172.21.15.33:0/3690175390 >> v1:172.21.15.33:6789/0 conn(0x7f6b500b46b0 legacy=0x7f6b500b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.980+0000 7f6b4e800640 1 -- 172.21.15.33:0/3690175390 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500ba020 con 0x7f6b500b56e0 2024-03-28T01:06:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.982+0000 7f6b4e800640 1 --2- 172.21.15.33:0/3690175390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b500b56e0 0x7f6b50141110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b400097a0 tx=0x7f6b4002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d4409ce26dcb0f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.982+0000 7f6b4d400640 1 -- 172.21.15.33:0/3690175390 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40031780 con 0x7f6b500b56e0 2024-03-28T01:06:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.982+0000 7f6b4d400640 1 -- 172.21.15.33:0/3690175390 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b40037070 con 0x7f6b500b56e0 2024-03-28T01:06:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.982+0000 7f6b4d400640 1 -- 172.21.15.33:0/3690175390 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40031b20 con 0x7f6b500b56e0 2024-03-28T01:06:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 -- 172.21.15.33:0/3690175390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b500b56e0 msgr2=0x7f6b50141110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 --2- 172.21.15.33:0/3690175390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b500b56e0 0x7f6b50141110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b400097a0 tx=0x7f6b4002b820 comp rx=0 tx=0).stop 2024-03-28T01:06:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 -- 172.21.15.33:0/3690175390 shutdown_connections 2024-03-28T01:06:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 --2- 172.21.15.33:0/3690175390 >> v2:172.21.15.194:3300/0 conn(0x7f6b50141650 0x7f6b5014a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 --2- 172.21.15.33:0/3690175390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b500b56e0 0x7f6b50141110 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 -- 172.21.15.33:0/3690175390 >> 172.21.15.33:0/3690175390 conn(0x7f6b50072970 msgr2=0x7f6b50072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 -- 172.21.15.33:0/3690175390 shutdown_connections 2024-03-28T01:06:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.983+0000 7f6b55400640 1 -- 172.21.15.33:0/3690175390 wait complete. 2024-03-28T01:06:25.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.984+0000 7f6b55400640 1 Processor -- start 2024-03-28T01:06:25.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b55400640 1 -- start start 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b55400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b500b46b0 0x7f6b500bd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b55400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500b56e0 0x7f6b50143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b55400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b50141650 0x7f6b501e0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b55400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b4270 con 0x7f6b500b56e0 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b55400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b40f0 con 0x7f6b500b46b0 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b55400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5014aec0 con 0x7f6b50141650 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.985+0000 7f6b4de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500b56e0 0x7f6b50143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:25.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500b56e0 0x7f6b50143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36218/0 (socket says 172.21.15.33:36218) 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 -- 172.21.15.33:0/1062384020 learned_addr learned my addr 172.21.15.33:0/1062384020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4e800640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b500b46b0 0x7f6b500bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4f200640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b50141650 0x7f6b501e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 -- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b50141650 msgr2=0x7f6b501e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b50141650 0x7f6b501e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 -- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b500b46b0 msgr2=0x7f6b500bd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b500b46b0 0x7f6b500bd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 -- 172.21.15.33:0/1062384020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500bd860 con 0x7f6b500b56e0 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4e800640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b500b46b0 0x7f6b500bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4f200640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b50141650 0x7f6b501e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.986+0000 7f6b4de00640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500b56e0 0x7f6b50143f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00e870 tx=0x7f6b3c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.987+0000 7f6b46a00640 1 -- 172.21.15.33:0/1062384020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c017070 con 0x7f6b500b56e0 2024-03-28T01:06:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.987+0000 7f6b46a00640 1 -- 172.21.15.33:0/1062384020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c00c940 con 0x7f6b500b56e0 2024-03-28T01:06:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.987+0000 7f6b46a00640 1 -- 172.21.15.33:0/1062384020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00cd30 con 0x7f6b500b56e0 2024-03-28T01:06:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.987+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b500bc8b0 con 0x7f6b500b56e0 2024-03-28T01:06:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.987+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b500bb220 con 0x7f6b500b56e0 2024-03-28T01:06:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.988+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b50075380 con 0x7f6b500b56e0 2024-03-28T01:06:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.989+0000 7f6b46a00640 1 -- 172.21.15.33:0/1062384020 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6b3c01a070 con 0x7f6b500b56e0 2024-03-28T01:06:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.990+0000 7f6b46a00640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f6b3006bbd0 0x7f6b3006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.990+0000 7f6b46a00640 1 -- 172.21.15.33:0/1062384020 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b3c08a8b0 con 0x7f6b500b56e0 2024-03-28T01:06:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.990+0000 7f6b4e800640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f6b3006bbd0 0x7f6b3006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.991+0000 7f6b4e800640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f6b3006bbd0 0x7f6b3006e090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b40036000 tx=0x7f6b40008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:25.999+0000 7f6b46a00640 1 -- 172.21.15.33:0/1062384020 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f6b3c059430 con 0x7f6b500b56e0 2024-03-28T01:06:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.337+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b501491d0 con 0x7f6b500b56e0 2024-03-28T01:06:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.338+0000 7f6b46a00640 1 -- 172.21.15.33:0/1062384020 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+3391 (secure 0 0 0) 0x7f6b3c058d30 con 0x7f6b500b56e0 2024-03-28T01:06:26.341 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:26.341 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/214301400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":214301400},{"type":"v1","addr":"172.21.15.194:6833","nonce":214301400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T01:06:22.173766+0000","modified":"2024-03-28T01:06:25.239216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2290842450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2290842450},{"type":"v1","addr":"172.21.15.33:6835","nonce":2290842450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-03-28T01:06:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-03-28T01:06:26.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f6b3006bbd0 msgr2=0x7f6b3006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f6b3006bbd0 0x7f6b3006e090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b40036000 tx=0x7f6b40008030 comp rx=0 tx=0).stop 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500b56e0 msgr2=0x7f6b50143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500b56e0 0x7f6b50143f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00e870 tx=0x7f6b3c00ed30 comp rx=0 tx=0).stop 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 shutdown_connections 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f6b3006bbd0 0x7f6b3006e090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b50141650 0x7f6b501e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500b56e0 0x7f6b50143f00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 --2- 172.21.15.33:0/1062384020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b500b46b0 0x7f6b500bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 >> 172.21.15.33:0/1062384020 conn(0x7f6b50072970 msgr2=0x7f6b50149f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 shutdown_connections 2024-03-28T01:06:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.341+0000 7f6b55400640 1 -- 172.21.15.33:0/1062384020 wait complete. 2024-03-28T01:06:26.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:26.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.597+0000 7fb809a00640 1 Processor -- start 2024-03-28T01:06:26.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.597+0000 7fb809a00640 1 -- start start 2024-03-28T01:06:26.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb809a00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fb804067040 0x7fb8040af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:26.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb809a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb80414b060 0x7fb804141300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:26.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb809a00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040aed00 con 0x7fb804140ec0 2024-03-28T01:06:26.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb809a00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb804067420 con 0x7fb80414b060 2024-03-28T01:06:26.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb809a00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040afa00 con 0x7fb804067040 2024-03-28T01:06:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb7fb400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb804140ec0 0x7fb80414a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41002/0 (socket says 172.21.15.33:41002) 2024-03-28T01:06:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb7fb400640 1 -- 172.21.15.33:0/729169357 learned_addr learned my addr 172.21.15.33:0/729169357 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.599+0000 7fb809000640 1 --2- 172.21.15.33:0/729169357 >> v2:172.21.15.33:3300/0 conn(0x7fb80414b060 0x7fb804141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.600+0000 7fb803e00640 1 --2- 172.21.15.33:0/729169357 >> v2:172.21.15.194:3300/0 conn(0x7fb804067040 0x7fb8040af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.600+0000 7fb803400640 1 -- 172.21.15.33:0/729169357 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375667926 0 0) 0x7fb8040aed00 con 0x7fb804140ec0 2024-03-28T01:06:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.600+0000 7fb803400640 1 -- 172.21.15.33:0/729169357 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e8003520 con 0x7fb804140ec0 2024-03-28T01:06:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.600+0000 7fb809000640 1 -- 172.21.15.33:0/729169357 >> v2:172.21.15.194:3300/0 conn(0x7fb804067040 msgr2=0x7fb8040af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:26.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.600+0000 7fb809000640 1 --2- 172.21.15.33:0/729169357 >> v2:172.21.15.194:3300/0 conn(0x7fb804067040 0x7fb8040af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.600+0000 7fb809000640 1 -- 172.21.15.33:0/729169357 >> v1:172.21.15.33:6789/0 conn(0x7fb804140ec0 legacy=0x7fb80414a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.600+0000 7fb809000640 1 -- 172.21.15.33:0/729169357 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040afce0 con 0x7fb80414b060 2024-03-28T01:06:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.602+0000 7fb809000640 1 --2- 172.21.15.33:0/729169357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80414b060 0x7fb804141300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb804067630 tx=0x7fb7fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba3b1f3e16409c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.602+0000 7fb803400640 1 -- 172.21.15.33:0/729169357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc035020 con 0x7fb80414b060 2024-03-28T01:06:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.602+0000 7fb803400640 1 -- 172.21.15.33:0/729169357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc031830 con 0x7fb80414b060 2024-03-28T01:06:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.602+0000 7fb803400640 1 -- 172.21.15.33:0/729169357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc031c00 con 0x7fb80414b060 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.602+0000 7fb809a00640 1 -- 172.21.15.33:0/729169357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80414b060 msgr2=0x7fb804141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.602+0000 7fb809a00640 1 --2- 172.21.15.33:0/729169357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80414b060 0x7fb804141300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb804067630 tx=0x7fb7fc02b8d0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.603+0000 7fb809a00640 1 -- 172.21.15.33:0/729169357 shutdown_connections 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.603+0000 7fb809a00640 1 --2- 172.21.15.33:0/729169357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80414b060 0x7fb804141300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.603+0000 7fb809a00640 1 --2- 172.21.15.33:0/729169357 >> v2:172.21.15.194:3300/0 conn(0x7fb804067040 0x7fb8040af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.603+0000 7fb809a00640 1 -- 172.21.15.33:0/729169357 >> 172.21.15.33:0/729169357 conn(0x7fb8040b5ac0 msgr2=0x7fb8040b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.603+0000 7fb809a00640 1 -- 172.21.15.33:0/729169357 shutdown_connections 2024-03-28T01:06:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.603+0000 7fb809a00640 1 -- 172.21.15.33:0/729169357 wait complete. 2024-03-28T01:06:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.604+0000 7fb809a00640 1 Processor -- start 2024-03-28T01:06:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.604+0000 7fb809a00640 1 -- start start 2024-03-28T01:06:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb809a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb804067040 0x7fb8040b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb809a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb804140ec0 0x7fb8041e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb809a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb80414b060 0x7fb8040b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb809a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040b5740 con 0x7fb804067040 2024-03-28T01:06:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb809a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040b4fc0 con 0x7fb804140ec0 2024-03-28T01:06:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb809a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040b58c0 con 0x7fb80414b060 2024-03-28T01:06:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb803e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb804067040 0x7fb8040b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb803e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb804067040 0x7fb8040b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36230/0 (socket says 172.21.15.33:36230) 2024-03-28T01:06:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.605+0000 7fb803e00640 1 -- 172.21.15.33:0/1281296923 learned_addr learned my addr 172.21.15.33:0/1281296923 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb7fb400640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb804140ec0 0x7fb8041e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb809000640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb80414b060 0x7fb8040b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb803e00640 1 -- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb80414b060 msgr2=0x7fb8040b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb803e00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb80414b060 0x7fb8040b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb803e00640 1 -- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb804140ec0 msgr2=0x7fb8041e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb803e00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb804140ec0 0x7fb8041e0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb803e00640 1 -- 172.21.15.33:0/1281296923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8041e0280 con 0x7fb804067040 2024-03-28T01:06:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb803e00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb804067040 0x7fb8040b0630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4007ec0 tx=0x7fb7f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.606+0000 7fb800c00640 1 -- 172.21.15.33:0/1281296923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4014620 con 0x7fb804067040 2024-03-28T01:06:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.607+0000 7fb800c00640 1 -- 172.21.15.33:0/1281296923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4019070 con 0x7fb804067040 2024-03-28T01:06:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.607+0000 7fb800c00640 1 -- 172.21.15.33:0/1281296923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4014a20 con 0x7fb804067040 2024-03-28T01:06:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.607+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8041e0560 con 0x7fb804067040 2024-03-28T01:06:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.607+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb804147290 con 0x7fb804067040 2024-03-28T01:06:26.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.608+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8041e0070 con 0x7fb804067040 2024-03-28T01:06:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.609+0000 7fb800c00640 1 -- 172.21.15.33:0/1281296923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb7f4014bc0 con 0x7fb804067040 2024-03-28T01:06:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.610+0000 7fb800c00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb7e006bbd0 0x7fb7e006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.610+0000 7fb800c00640 1 -- 172.21.15.33:0/1281296923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7f408b430 con 0x7fb804067040 2024-03-28T01:06:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.610+0000 7fb7fb400640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb7e006bbd0 0x7fb7e006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.611+0000 7fb7fb400640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb7e006bbd0 0x7fb7e006e090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb80409eff0 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.615+0000 7fb800c00640 1 -- 172.21.15.33:0/1281296923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb7f4059fb0 con 0x7fb804067040 2024-03-28T01:06:26.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.821+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb804075310 con 0x7fb804067040 2024-03-28T01:06:26.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.823+0000 7fb800c00640 1 -- 172.21.15.33:0/1281296923 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fb7f40598b0 con 0x7fb804067040 2024-03-28T01:06:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:26.826 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:23.710537+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb7e006bbd0 msgr2=0x7fb7e006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb7e006bbd0 0x7fb7e006e090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb80409eff0 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb804067040 msgr2=0x7fb8040b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb804067040 0x7fb8040b0630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4007ec0 tx=0x7fb7f400e4b0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 shutdown_connections 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb7e006bbd0 0x7fb7e006e090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb80414b060 0x7fb8040b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb804140ec0 0x7fb8041e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 --2- 172.21.15.33:0/1281296923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb804067040 0x7fb8040b0630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 >> 172.21.15.33:0/1281296923 conn(0x7fb8040b5ac0 msgr2=0x7fb80409aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 shutdown_connections 2024-03-28T01:06:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:26.826+0000 7fb809a00640 1 -- 172.21.15.33:0/1281296923 wait complete. 2024-03-28T01:06:26.851 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T01:06:26.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:06:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.055+0000 7f8b75600640 1 Processor -- start 2024-03-28T01:06:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.056+0000 7f8b75600640 1 -- start start 2024-03-28T01:06:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.057+0000 7f8b75600640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b700b8d00 0x7f8b7014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.057+0000 7f8b75600640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700b55b0 con 0x7f8b700b51d0 2024-03-28T01:06:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.057+0000 7f8b75600640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700b9190 con 0x7f8b700b6b80 2024-03-28T01:06:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.057+0000 7f8b75600640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7014a820 con 0x7f8b700b8d00 2024-03-28T01:06:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.057+0000 7f8b6fe00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b700b51d0 0x7f8b700b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41008/0 (socket says 172.21.15.33:41008) 2024-03-28T01:06:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.057+0000 7f8b6fe00640 1 -- 172.21.15.33:0/2932631867 learned_addr learned my addr 172.21.15.33:0/2932631867 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.057+0000 7f8b67e00640 1 --2- 172.21.15.33:0/2932631867 >> v2:172.21.15.33:3300/0 conn(0x7f8b700b8d00 0x7f8b7014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.058+0000 7f8b6f400640 1 -- 172.21.15.33:0/2932631867 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390094213 0 0) 0x7f8b700b55b0 con 0x7f8b700b51d0 2024-03-28T01:06:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.058+0000 7f8b6f400640 1 -- 172.21.15.33:0/2932631867 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b54003520 con 0x7f8b700b51d0 2024-03-28T01:06:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.058+0000 7f8b67e00640 1 -- 172.21.15.33:0/2932631867 >> v1:172.21.15.194:6789/0 conn(0x7f8b700b6b80 legacy=0x7f8b7014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.058+0000 7f8b67e00640 1 -- 172.21.15.33:0/2932631867 >> v1:172.21.15.33:6789/0 conn(0x7f8b700b51d0 legacy=0x7f8b700b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.058+0000 7f8b67e00640 1 -- 172.21.15.33:0/2932631867 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700b9800 con 0x7f8b700b8d00 2024-03-28T01:06:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.060+0000 7f8b67e00640 1 --2- 172.21.15.33:0/2932631867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b8d00 0x7f8b7014ac70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b580097a0 tx=0x7f8b5802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a1728f764e55f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.060+0000 7f8b6f400640 1 -- 172.21.15.33:0/2932631867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58034070 con 0x7f8b700b8d00 2024-03-28T01:06:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.060+0000 7f8b6f400640 1 -- 172.21.15.33:0/2932631867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b58030610 con 0x7f8b700b8d00 2024-03-28T01:06:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.060+0000 7f8b6f400640 1 -- 172.21.15.33:0/2932631867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580309e0 con 0x7f8b700b8d00 2024-03-28T01:06:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.061+0000 7f8b75600640 1 -- 172.21.15.33:0/2932631867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b8d00 msgr2=0x7f8b7014ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.061+0000 7f8b75600640 1 --2- 172.21.15.33:0/2932631867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b8d00 0x7f8b7014ac70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b580097a0 tx=0x7f8b5802b960 comp rx=0 tx=0).stop 2024-03-28T01:06:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.061+0000 7f8b75600640 1 -- 172.21.15.33:0/2932631867 shutdown_connections 2024-03-28T01:06:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.061+0000 7f8b75600640 1 --2- 172.21.15.33:0/2932631867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b8d00 0x7f8b7014ac70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.061+0000 7f8b75600640 1 -- 172.21.15.33:0/2932631867 >> 172.21.15.33:0/2932631867 conn(0x7f8b70072940 msgr2=0x7f8b70072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.061+0000 7f8b75600640 1 -- 172.21.15.33:0/2932631867 shutdown_connections 2024-03-28T01:06:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.061+0000 7f8b75600640 1 -- 172.21.15.33:0/2932631867 wait complete. 2024-03-28T01:06:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.063+0000 7f8b75600640 1 Processor -- start 2024-03-28T01:06:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.063+0000 7f8b75600640 1 -- start start 2024-03-28T01:06:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b75600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b51d0 0x7f8b7013bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b75600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b700b6b80 0x7f8b701436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b75600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b7013dad0 0x7f8b70141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b75600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700b39c0 con 0x7f8b700b51d0 2024-03-28T01:06:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b75600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700b44c0 con 0x7f8b7013dad0 2024-03-28T01:06:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b75600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b70152b80 con 0x7f8b700b6b80 2024-03-28T01:06:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b51d0 0x7f8b7013bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b51d0 0x7f8b7013bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36242/0 (socket says 172.21.15.33:36242) 2024-03-28T01:06:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.064+0000 7f8b6fe00640 1 -- 172.21.15.33:0/3696969431 learned_addr learned my addr 172.21.15.33:0/3696969431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6fe00640 1 -- 172.21.15.33:0/3696969431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b700b6b80 msgr2=0x7f8b701436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6fe00640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b700b6b80 0x7f8b701436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6fe00640 1 -- 172.21.15.33:0/3696969431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b7013dad0 msgr2=0x7f8b70141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6fe00640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b7013dad0 0x7f8b70141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6fe00640 1 -- 172.21.15.33:0/3696969431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7013e000 con 0x7f8b700b51d0 2024-03-28T01:06:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6fe00640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b51d0 0x7f8b7013bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000c430 tx=0x7f8b6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6cc00640 1 -- 172.21.15.33:0/3696969431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60017070 con 0x7f8b700b51d0 2024-03-28T01:06:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6cc00640 1 -- 172.21.15.33:0/3696969431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b60005ce0 con 0x7f8b700b51d0 2024-03-28T01:06:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b6cc00640 1 -- 172.21.15.33:0/3696969431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b600072f0 con 0x7f8b700b51d0 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.065+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b70144900 con 0x7f8b700b51d0 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.066+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b70142310 con 0x7f8b700b51d0 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.067+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b70075310 con 0x7f8b700b51d0 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.068+0000 7f8b6cc00640 1 -- 172.21.15.33:0/3696969431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8b6001a070 con 0x7f8b700b51d0 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.068+0000 7f8b6cc00640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8b4c06bbd0 0x7f8b4c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.069+0000 7f8b6cc00640 1 -- 172.21.15.33:0/3696969431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b60089f30 con 0x7f8b700b51d0 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.069+0000 7f8b67e00640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8b4c06bbd0 0x7f8b4c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.069+0000 7f8b67e00640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8b4c06bbd0 0x7f8b4c06e090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8b58036000 tx=0x7f8b58030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.073+0000 7f8b6cc00640 1 -- 172.21.15.33:0/3696969431 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8b7013e000 con 0x7f8b700b51d0 2024-03-28T01:06:27.093 INFO:tasks.ceph.mds.a.smithi033.stderr:2024-03-28T01:06:27.090+0000 7f093ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T01:06:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.271+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b70143be0 con 0x7f8b4c06bbd0 2024-03-28T01:06:27.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.280+0000 7f8b6cc00640 1 -- 172.21.15.33:0/3696969431 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251920 (secure 0 0 0) 0x7f8b70143be0 con 0x7f8b4c06bbd0 2024-03-28T01:06:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:06:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8b4c06bbd0 msgr2=0x7f8b4c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8b4c06bbd0 0x7f8b4c06e090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8b58036000 tx=0x7f8b58030000 comp rx=0 tx=0).stop 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b51d0 msgr2=0x7f8b7013bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b51d0 0x7f8b7013bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000c430 tx=0x7f8b6000c8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 shutdown_connections 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8b4c06bbd0 0x7f8b4c06e090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b7013dad0 0x7f8b70141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b700b6b80 0x7f8b701436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 --2- 172.21.15.33:0/3696969431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b700b51d0 0x7f8b7013bda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 >> 172.21.15.33:0/3696969431 conn(0x7f8b70072940 msgr2=0x7f8b70145ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 shutdown_connections 2024-03-28T01:06:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.285+0000 7f8b75600640 1 -- 172.21.15.33:0/3696969431 wait complete. 2024-03-28T01:06:27.305 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-28T01:06:26.385140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.711491"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750987+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:06:22.750987+0000","last_peered":"2024-03-28T01:06:22.750987+0000","last_clean":"2024-03-28T01:06:22.750987+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:06:22.750987+0000","last_undegraded":"2024-03-28T01:06:22.750987+0000","last_fullsized":"2024-03-28T01:06:22.750987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711746+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:06:24.711746+0000","last_peered":"2024-03-28T01:06:24.711746+0000","last_clean":"2024-03-28T01:06:24.711746+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:06:24.711746+0000","last_undegraded":"2024-03-28T01:06:24.711746+0000","last_fullsized":"2024-03-28T01:06:24.711746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723513+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:06:23.723513+0000","last_peered":"2024-03-28T01:06:23.723513+0000","last_clean":"2024-03-28T01:06:23.723513+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:06:23.723513+0000","last_undegraded":"2024-03-28T01:06:23.723513+0000","last_fullsized":"2024-03-28T01:06:23.723513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.754630+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:06:22.754630+0000","last_peered":"2024-03-28T01:06:22.754630+0000","last_clean":"2024-03-28T01:06:22.754630+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:06:22.754630+0000","last_undegraded":"2024-03-28T01:06:22.754630+0000","last_fullsized":"2024-03-28T01:06:22.754630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750828+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:06:22.750828+0000","last_peered":"2024-03-28T01:06:22.750828+0000","last_clean":"2024-03-28T01:06:22.750828+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:06:22.750828+0000","last_undegraded":"2024-03-28T01:06:22.750828+0000","last_fullsized":"2024-03-28T01:06:22.750828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.744777+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:06:22.744777+0000","last_peered":"2024-03-28T01:06:22.744777+0000","last_clean":"2024-03-28T01:06:22.744777+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:06:22.744777+0000","last_undegraded":"2024-03-28T01:06:22.744777+0000","last_fullsized":"2024-03-28T01:06:22.744777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.749623+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:06:22.749623+0000","last_peered":"2024-03-28T01:06:22.749623+0000","last_clean":"2024-03-28T01:06:22.749623+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:06:22.749623+0000","last_undegraded":"2024-03-28T01:06:22.749623+0000","last_fullsized":"2024-03-28T01:06:22.749623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723763+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:06:23.723763+0000","last_peered":"2024-03-28T01:06:23.723763+0000","last_clean":"2024-03-28T01:06:23.723763+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:06:23.723763+0000","last_undegraded":"2024-03-28T01:06:23.723763+0000","last_fullsized":"2024-03-28T01:06:23.723763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750967+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:06:22.750967+0000","last_peered":"2024-03-28T01:06:22.750967+0000","last_clean":"2024-03-28T01:06:22.750967+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:06:22.750967+0000","last_undegraded":"2024-03-28T01:06:22.750967+0000","last_fullsized":"2024-03-28T01:06:22.750967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718027+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:06:24.718027+0000","last_peered":"2024-03-28T01:06:24.718027+0000","last_clean":"2024-03-28T01:06:24.718027+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:06:24.718027+0000","last_undegraded":"2024-03-28T01:06:24.718027+0000","last_fullsized":"2024-03-28T01:06:24.718027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717838+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:06:24.717838+0000","last_peered":"2024-03-28T01:06:24.717838+0000","last_clean":"2024-03-28T01:06:24.717838+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:06:24.717838+0000","last_undegraded":"2024-03-28T01:06:24.717838+0000","last_fullsized":"2024-03-28T01:06:24.717838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727289+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:06:23.727289+0000","last_peered":"2024-03-28T01:06:23.727289+0000","last_clean":"2024-03-28T01:06:23.727289+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:06:23.727289+0000","last_undegraded":"2024-03-28T01:06:23.727289+0000","last_fullsized":"2024-03-28T01:06:23.727289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724224+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:06:23.724224+0000","last_peered":"2024-03-28T01:06:23.724224+0000","last_clean":"2024-03-28T01:06:23.724224+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:06:23.724224+0000","last_undegraded":"2024-03-28T01:06:23.724224+0000","last_fullsized":"2024-03-28T01:06:23.724224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722772+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:06:23.722772+0000","last_peered":"2024-03-28T01:06:23.722772+0000","last_clean":"2024-03-28T01:06:23.722772+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:06:23.722772+0000","last_undegraded":"2024-03-28T01:06:23.722772+0000","last_fullsized":"2024-03-28T01:06:23.722772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710748+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:06:24.710748+0000","last_peered":"2024-03-28T01:06:24.710748+0000","last_clean":"2024-03-28T01:06:24.710748+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:06:24.710748+0000","last_undegraded":"2024-03-28T01:06:24.710748+0000","last_fullsized":"2024-03-28T01:06:24.710748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.744683+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:06:22.744683+0000","last_peered":"2024-03-28T01:06:22.744683+0000","last_clean":"2024-03-28T01:06:22.744683+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:06:22.744683+0000","last_undegraded":"2024-03-28T01:06:22.744683+0000","last_fullsized":"2024-03-28T01:06:22.744683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752236+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:06:22.752236+0000","last_peered":"2024-03-28T01:06:22.752236+0000","last_clean":"2024-03-28T01:06:22.752236+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:06:22.752236+0000","last_undegraded":"2024-03-28T01:06:22.752236+0000","last_fullsized":"2024-03-28T01:06:22.752236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.742233+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:06:22.742233+0000","last_peered":"2024-03-28T01:06:22.742233+0000","last_clean":"2024-03-28T01:06:22.742233+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:06:22.742233+0000","last_undegraded":"2024-03-28T01:06:22.742233+0000","last_fullsized":"2024-03-28T01:06:22.742233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746843+0000","last_change":"2024-03-28T01:06:22.743819+0000","last_active":"2024-03-28T01:06:22.746843+0000","last_peered":"2024-03-28T01:06:22.746843+0000","last_clean":"2024-03-28T01:06:22.746843+0000","last_became_active":"2024-03-28T01:06:22.742898+0000","last_became_peered":"2024-03-28T01:06:22.742898+0000","last_unstale":"2024-03-28T01:06:22.746843+0000","last_undegraded":"2024-03-28T01:06:22.746843+0000","last_fullsized":"2024-03-28T01:06:22.746843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717295+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:06:24.717295+0000","last_peered":"2024-03-28T01:06:24.717295+0000","last_clean":"2024-03-28T01:06:24.717295+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:06:24.717295+0000","last_undegraded":"2024-03-28T01:06:24.717295+0000","last_fullsized":"2024-03-28T01:06:24.717295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722929+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:06:23.722929+0000","last_peered":"2024-03-28T01:06:23.722929+0000","last_clean":"2024-03-28T01:06:23.722929+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:06:23.722929+0000","last_undegraded":"2024-03-28T01:06:23.722929+0000","last_fullsized":"2024-03-28T01:06:23.722929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752340+0000","last_change":"2024-03-28T01:06:22.749132+0000","last_active":"2024-03-28T01:06:22.752340+0000","last_peered":"2024-03-28T01:06:22.752340+0000","last_clean":"2024-03-28T01:06:22.752340+0000","last_became_active":"2024-03-28T01:06:22.748012+0000","last_became_peered":"2024-03-28T01:06:22.748012+0000","last_unstale":"2024-03-28T01:06:22.752340+0000","last_undegraded":"2024-03-28T01:06:22.752340+0000","last_fullsized":"2024-03-28T01:06:22.752340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724138+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:06:23.724138+0000","last_peered":"2024-03-28T01:06:23.724138+0000","last_clean":"2024-03-28T01:06:23.724138+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:06:23.724138+0000","last_undegraded":"2024-03-28T01:06:23.724138+0000","last_fullsized":"2024-03-28T01:06:23.724138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752128+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:06:22.752128+0000","last_peered":"2024-03-28T01:06:22.752128+0000","last_clean":"2024-03-28T01:06:22.752128+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:06:22.752128+0000","last_undegraded":"2024-03-28T01:06:22.752128+0000","last_fullsized":"2024-03-28T01:06:22.752128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724983+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:06:23.724983+0000","last_peered":"2024-03-28T01:06:23.724983+0000","last_clean":"2024-03-28T01:06:23.724983+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:06:23.724983+0000","last_undegraded":"2024-03-28T01:06:23.724983+0000","last_fullsized":"2024-03-28T01:06:23.724983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718057+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:06:24.718057+0000","last_peered":"2024-03-28T01:06:24.718057+0000","last_clean":"2024-03-28T01:06:24.718057+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:06:24.718057+0000","last_undegraded":"2024-03-28T01:06:24.718057+0000","last_fullsized":"2024-03-28T01:06:24.718057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711534+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:06:24.711534+0000","last_peered":"2024-03-28T01:06:24.711534+0000","last_clean":"2024-03-28T01:06:24.711534+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:06:24.711534+0000","last_undegraded":"2024-03-28T01:06:24.711534+0000","last_fullsized":"2024-03-28T01:06:24.711534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725154+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:06:23.725154+0000","last_peered":"2024-03-28T01:06:23.725154+0000","last_clean":"2024-03-28T01:06:23.725154+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:06:23.725154+0000","last_undegraded":"2024-03-28T01:06:23.725154+0000","last_fullsized":"2024-03-28T01:06:23.725154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723116+0000","last_change":"2024-03-28T01:06:22.731272+0000","last_active":"2024-03-28T01:06:23.723116+0000","last_peered":"2024-03-28T01:06:23.723116+0000","last_clean":"2024-03-28T01:06:23.723116+0000","last_became_active":"2024-03-28T01:06:22.730261+0000","last_became_peered":"2024-03-28T01:06:22.730261+0000","last_unstale":"2024-03-28T01:06:23.723116+0000","last_undegraded":"2024-03-28T01:06:23.723116+0000","last_fullsized":"2024-03-28T01:06:23.723116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752328+0000","last_change":"2024-03-28T01:06:22.746590+0000","last_active":"2024-03-28T01:06:22.752328+0000","last_peered":"2024-03-28T01:06:22.752328+0000","last_clean":"2024-03-28T01:06:22.752328+0000","last_became_active":"2024-03-28T01:06:22.746299+0000","last_became_peered":"2024-03-28T01:06:22.746299+0000","last_unstale":"2024-03-28T01:06:22.752328+0000","last_undegraded":"2024-03-28T01:06:22.752328+0000","last_fullsized":"2024-03-28T01:06:22.752328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.739711+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:06:22.739711+0000","last_peered":"2024-03-28T01:06:22.739711+0000","last_clean":"2024-03-28T01:06:22.739711+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:06:22.739711+0000","last_undegraded":"2024-03-28T01:06:22.739711+0000","last_fullsized":"2024-03-28T01:06:22.739711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717294+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:06:24.717294+0000","last_peered":"2024-03-28T01:06:24.717294+0000","last_clean":"2024-03-28T01:06:24.717294+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:06:24.717294+0000","last_undegraded":"2024-03-28T01:06:24.717294+0000","last_fullsized":"2024-03-28T01:06:24.717294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726504+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:06:23.726504+0000","last_peered":"2024-03-28T01:06:23.726504+0000","last_clean":"2024-03-28T01:06:23.726504+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:06:23.726504+0000","last_undegraded":"2024-03-28T01:06:23.726504+0000","last_fullsized":"2024-03-28T01:06:23.726504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724518+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:06:23.724518+0000","last_peered":"2024-03-28T01:06:23.724518+0000","last_clean":"2024-03-28T01:06:23.724518+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:06:23.724518+0000","last_undegraded":"2024-03-28T01:06:23.724518+0000","last_fullsized":"2024-03-28T01:06:23.724518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723265+0000","last_change":"2024-03-28T01:06:22.717986+0000","last_active":"2024-03-28T01:06:23.723265+0000","last_peered":"2024-03-28T01:06:23.723265+0000","last_clean":"2024-03-28T01:06:23.723265+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:06:23.723265+0000","last_undegraded":"2024-03-28T01:06:23.723265+0000","last_fullsized":"2024-03-28T01:06:23.723265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717592+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:06:24.717592+0000","last_peered":"2024-03-28T01:06:24.717592+0000","last_clean":"2024-03-28T01:06:24.717592+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:06:24.717592+0000","last_undegraded":"2024-03-28T01:06:24.717592+0000","last_fullsized":"2024-03-28T01:06:24.717592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724278+0000","last_change":"2024-03-28T01:06:21.721278+0000","last_active":"2024-03-28T01:06:23.724278+0000","last_peered":"2024-03-28T01:06:23.724278+0000","last_clean":"2024-03-28T01:06:23.724278+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:06:23.724278+0000","last_undegraded":"2024-03-28T01:06:23.724278+0000","last_fullsized":"2024-03-28T01:06:23.724278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723815+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:06:23.723815+0000","last_peered":"2024-03-28T01:06:23.723815+0000","last_clean":"2024-03-28T01:06:23.723815+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:06:23.723815+0000","last_undegraded":"2024-03-28T01:06:23.723815+0000","last_fullsized":"2024-03-28T01:06:23.723815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.727997+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:06:22.727997+0000","last_peered":"2024-03-28T01:06:22.727997+0000","last_clean":"2024-03-28T01:06:22.727997+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:06:22.727997+0000","last_undegraded":"2024-03-28T01:06:22.727997+0000","last_fullsized":"2024-03-28T01:06:22.727997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710729+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:06:24.710729+0000","last_peered":"2024-03-28T01:06:24.710729+0000","last_clean":"2024-03-28T01:06:24.710729+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:06:24.710729+0000","last_undegraded":"2024-03-28T01:06:24.710729+0000","last_fullsized":"2024-03-28T01:06:24.710729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717704+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:06:24.717704+0000","last_peered":"2024-03-28T01:06:24.717704+0000","last_clean":"2024-03-28T01:06:24.717704+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:06:24.717704+0000","last_undegraded":"2024-03-28T01:06:24.717704+0000","last_fullsized":"2024-03-28T01:06:24.717704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752268+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:06:22.752268+0000","last_peered":"2024-03-28T01:06:22.752268+0000","last_clean":"2024-03-28T01:06:22.752268+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:06:22.752268+0000","last_undegraded":"2024-03-28T01:06:22.752268+0000","last_fullsized":"2024-03-28T01:06:22.752268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.731365+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:06:22.731365+0000","last_peered":"2024-03-28T01:06:22.731365+0000","last_clean":"2024-03-28T01:06:22.731365+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:06:22.731365+0000","last_undegraded":"2024-03-28T01:06:22.731365+0000","last_fullsized":"2024-03-28T01:06:22.731365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724058+0000","last_change":"2024-03-28T01:06:22.743892+0000","last_active":"2024-03-28T01:06:23.724058+0000","last_peered":"2024-03-28T01:06:23.724058+0000","last_clean":"2024-03-28T01:06:23.724058+0000","last_became_active":"2024-03-28T01:06:22.743113+0000","last_became_peered":"2024-03-28T01:06:22.743113+0000","last_unstale":"2024-03-28T01:06:23.724058+0000","last_undegraded":"2024-03-28T01:06:23.724058+0000","last_fullsized":"2024-03-28T01:06:23.724058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711550+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:06:24.711550+0000","last_peered":"2024-03-28T01:06:24.711550+0000","last_clean":"2024-03-28T01:06:24.711550+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:06:24.711550+0000","last_undegraded":"2024-03-28T01:06:24.711550+0000","last_fullsized":"2024-03-28T01:06:24.711550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727568+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:06:23.727568+0000","last_peered":"2024-03-28T01:06:23.727568+0000","last_clean":"2024-03-28T01:06:23.727568+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:06:23.727568+0000","last_undegraded":"2024-03-28T01:06:23.727568+0000","last_fullsized":"2024-03-28T01:06:23.727568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717385+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:06:24.717385+0000","last_peered":"2024-03-28T01:06:24.717385+0000","last_clean":"2024-03-28T01:06:24.717385+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:06:24.717385+0000","last_undegraded":"2024-03-28T01:06:24.717385+0000","last_fullsized":"2024-03-28T01:06:24.717385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746820+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:06:22.746820+0000","last_peered":"2024-03-28T01:06:22.746820+0000","last_clean":"2024-03-28T01:06:22.746820+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:06:22.746820+0000","last_undegraded":"2024-03-28T01:06:22.746820+0000","last_fullsized":"2024-03-28T01:06:22.746820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723346+0000","last_change":"2024-03-28T01:06:21.727305+0000","last_active":"2024-03-28T01:06:23.723346+0000","last_peered":"2024-03-28T01:06:23.723346+0000","last_clean":"2024-03-28T01:06:23.723346+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:06:23.723346+0000","last_undegraded":"2024-03-28T01:06:23.723346+0000","last_fullsized":"2024-03-28T01:06:23.723346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722951+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:06:23.722951+0000","last_peered":"2024-03-28T01:06:23.722951+0000","last_clean":"2024-03-28T01:06:23.722951+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:06:23.722951+0000","last_undegraded":"2024-03-28T01:06:23.722951+0000","last_fullsized":"2024-03-28T01:06:23.722951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.729397+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:06:22.729397+0000","last_peered":"2024-03-28T01:06:22.729397+0000","last_clean":"2024-03-28T01:06:22.729397+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:06:22.729397+0000","last_undegraded":"2024-03-28T01:06:22.729397+0000","last_fullsized":"2024-03-28T01:06:22.729397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746354+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:06:22.746354+0000","last_peered":"2024-03-28T01:06:22.746354+0000","last_clean":"2024-03-28T01:06:22.746354+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:06:22.746354+0000","last_undegraded":"2024-03-28T01:06:22.746354+0000","last_fullsized":"2024-03-28T01:06:22.746354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:09:36.130037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717022+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:06:24.717022+0000","last_peered":"2024-03-28T01:06:24.717022+0000","last_clean":"2024-03-28T01:06:24.717022+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:06:24.717022+0000","last_undegraded":"2024-03-28T01:06:24.717022+0000","last_fullsized":"2024-03-28T01:06:24.717022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717552+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:06:24.717552+0000","last_peered":"2024-03-28T01:06:24.717552+0000","last_clean":"2024-03-28T01:06:24.717552+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:06:24.717552+0000","last_undegraded":"2024-03-28T01:06:24.717552+0000","last_fullsized":"2024-03-28T01:06:24.717552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.738398+0000","last_change":"2024-03-28T01:06:21.719466+0000","last_active":"2024-03-28T01:06:22.738398+0000","last_peered":"2024-03-28T01:06:22.738398+0000","last_clean":"2024-03-28T01:06:22.738398+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:06:22.738398+0000","last_undegraded":"2024-03-28T01:06:22.738398+0000","last_fullsized":"2024-03-28T01:06:22.738398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746930+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:06:22.746930+0000","last_peered":"2024-03-28T01:06:22.746930+0000","last_clean":"2024-03-28T01:06:22.746930+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:06:22.746930+0000","last_undegraded":"2024-03-28T01:06:22.746930+0000","last_fullsized":"2024-03-28T01:06:22.746930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727523+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:06:23.727523+0000","last_peered":"2024-03-28T01:06:23.727523+0000","last_clean":"2024-03-28T01:06:23.727523+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:06:23.727523+0000","last_undegraded":"2024-03-28T01:06:23.727523+0000","last_fullsized":"2024-03-28T01:06:23.727523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.749659+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:06:22.749659+0000","last_peered":"2024-03-28T01:06:22.749659+0000","last_clean":"2024-03-28T01:06:22.749659+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:06:22.749659+0000","last_undegraded":"2024-03-28T01:06:22.749659+0000","last_fullsized":"2024-03-28T01:06:22.749659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.742461+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:06:22.742461+0000","last_peered":"2024-03-28T01:06:22.742461+0000","last_clean":"2024-03-28T01:06:22.742461+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:06:22.742461+0000","last_undegraded":"2024-03-28T01:06:22.742461+0000","last_fullsized":"2024-03-28T01:06:22.742461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.755435+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:06:22.755435+0000","last_peered":"2024-03-28T01:06:22.755435+0000","last_clean":"2024-03-28T01:06:22.755435+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:06:22.755435+0000","last_undegraded":"2024-03-28T01:06:22.755435+0000","last_fullsized":"2024-03-28T01:06:22.755435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711878+0000","last_change":"2024-03-28T01:06:21.718159+0000","last_active":"2024-03-28T01:06:24.711878+0000","last_peered":"2024-03-28T01:06:24.711878+0000","last_clean":"2024-03-28T01:06:24.711878+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:06:24.711878+0000","last_undegraded":"2024-03-28T01:06:24.711878+0000","last_fullsized":"2024-03-28T01:06:24.711878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.754652+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:06:22.754652+0000","last_peered":"2024-03-28T01:06:22.754652+0000","last_clean":"2024-03-28T01:06:22.754652+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:06:22.754652+0000","last_undegraded":"2024-03-28T01:06:22.754652+0000","last_fullsized":"2024-03-28T01:06:22.754652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.720119+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:06:22.720119+0000","last_peered":"2024-03-28T01:06:22.720119+0000","last_clean":"2024-03-28T01:06:22.720119+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:06:22.720119+0000","last_undegraded":"2024-03-28T01:06:22.720119+0000","last_fullsized":"2024-03-28T01:06:22.720119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750924+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:06:22.750924+0000","last_peered":"2024-03-28T01:06:22.750924+0000","last_clean":"2024-03-28T01:06:22.750924+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:06:22.750924+0000","last_undegraded":"2024-03-28T01:06:22.750924+0000","last_fullsized":"2024-03-28T01:06:22.750924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.721860+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:06:22.721860+0000","last_peered":"2024-03-28T01:06:22.721860+0000","last_clean":"2024-03-28T01:06:22.721860+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:06:22.721860+0000","last_undegraded":"2024-03-28T01:06:22.721860+0000","last_fullsized":"2024-03-28T01:06:22.721860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.729015+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:06:22.729015+0000","last_peered":"2024-03-28T01:06:22.729015+0000","last_clean":"2024-03-28T01:06:22.729015+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:06:22.729015+0000","last_undegraded":"2024-03-28T01:06:22.729015+0000","last_fullsized":"2024-03-28T01:06:22.729015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724610+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:06:23.724610+0000","last_peered":"2024-03-28T01:06:23.724610+0000","last_clean":"2024-03-28T01:06:23.724610+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:06:23.724610+0000","last_undegraded":"2024-03-28T01:06:23.724610+0000","last_fullsized":"2024-03-28T01:06:23.724610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.721801+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:06:22.721801+0000","last_peered":"2024-03-28T01:06:22.721801+0000","last_clean":"2024-03-28T01:06:22.721801+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:06:22.721801+0000","last_undegraded":"2024-03-28T01:06:22.721801+0000","last_fullsized":"2024-03-28T01:06:22.721801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722723+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:06:23.722723+0000","last_peered":"2024-03-28T01:06:23.722723+0000","last_clean":"2024-03-28T01:06:23.722723+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:06:23.722723+0000","last_undegraded":"2024-03-28T01:06:23.722723+0000","last_fullsized":"2024-03-28T01:06:23.722723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.716574+0000","last_change":"2024-03-28T01:06:21.720753+0000","last_active":"2024-03-28T01:06:22.716574+0000","last_peered":"2024-03-28T01:06:22.716574+0000","last_clean":"2024-03-28T01:06:22.716574+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:06:22.716574+0000","last_undegraded":"2024-03-28T01:06:22.716574+0000","last_fullsized":"2024-03-28T01:06:22.716574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710942+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:06:24.710942+0000","last_peered":"2024-03-28T01:06:24.710942+0000","last_clean":"2024-03-28T01:06:24.710942+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:06:24.710942+0000","last_undegraded":"2024-03-28T01:06:24.710942+0000","last_fullsized":"2024-03-28T01:06:24.710942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.729552+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:06:22.729552+0000","last_peered":"2024-03-28T01:06:22.729552+0000","last_clean":"2024-03-28T01:06:22.729552+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:06:22.729552+0000","last_undegraded":"2024-03-28T01:06:22.729552+0000","last_fullsized":"2024-03-28T01:06:22.729552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727032+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:06:23.727032+0000","last_peered":"2024-03-28T01:06:23.727032+0000","last_clean":"2024-03-28T01:06:23.727032+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:06:23.727032+0000","last_undegraded":"2024-03-28T01:06:23.727032+0000","last_fullsized":"2024-03-28T01:06:23.727032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711510+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:06:24.711510+0000","last_peered":"2024-03-28T01:06:24.711510+0000","last_clean":"2024-03-28T01:06:24.711510+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:06:24.711510+0000","last_undegraded":"2024-03-28T01:06:24.711510+0000","last_fullsized":"2024-03-28T01:06:24.711510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752079+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:06:22.752079+0000","last_peered":"2024-03-28T01:06:22.752079+0000","last_clean":"2024-03-28T01:06:22.752079+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:06:22.752079+0000","last_undegraded":"2024-03-28T01:06:22.752079+0000","last_fullsized":"2024-03-28T01:06:22.752079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.725638+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:06:22.725638+0000","last_peered":"2024-03-28T01:06:22.725638+0000","last_clean":"2024-03-28T01:06:22.725638+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:06:22.725638+0000","last_undegraded":"2024-03-28T01:06:22.725638+0000","last_fullsized":"2024-03-28T01:06:22.725638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711037+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:06:24.711037+0000","last_peered":"2024-03-28T01:06:24.711037+0000","last_clean":"2024-03-28T01:06:24.711037+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:06:24.711037+0000","last_undegraded":"2024-03-28T01:06:24.711037+0000","last_fullsized":"2024-03-28T01:06:24.711037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.715740+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:06:24.715740+0000","last_peered":"2024-03-28T01:06:24.715740+0000","last_clean":"2024-03-28T01:06:24.715740+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:06:24.715740+0000","last_undegraded":"2024-03-28T01:06:24.715740+0000","last_fullsized":"2024-03-28T01:06:24.715740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:34.313143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725201+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:06:23.725201+0000","last_peered":"2024-03-28T01:06:23.725201+0000","last_clean":"2024-03-28T01:06:23.725201+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:06:23.725201+0000","last_undegraded":"2024-03-28T01:06:23.725201+0000","last_fullsized":"2024-03-28T01:06:23.725201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717736+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:06:24.717736+0000","last_peered":"2024-03-28T01:06:24.717736+0000","last_clean":"2024-03-28T01:06:24.717736+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:06:24.717736+0000","last_undegraded":"2024-03-28T01:06:24.717736+0000","last_fullsized":"2024-03-28T01:06:24.717736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723864+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:06:23.723864+0000","last_peered":"2024-03-28T01:06:23.723864+0000","last_clean":"2024-03-28T01:06:23.723864+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:06:23.723864+0000","last_undegraded":"2024-03-28T01:06:23.723864+0000","last_fullsized":"2024-03-28T01:06:23.723864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.721637+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:06:22.721637+0000","last_peered":"2024-03-28T01:06:22.721637+0000","last_clean":"2024-03-28T01:06:22.721637+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:06:22.721637+0000","last_undegraded":"2024-03-28T01:06:22.721637+0000","last_fullsized":"2024-03-28T01:06:22.721637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726343+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:06:23.726343+0000","last_peered":"2024-03-28T01:06:23.726343+0000","last_clean":"2024-03-28T01:06:23.726343+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:06:23.726343+0000","last_undegraded":"2024-03-28T01:06:23.726343+0000","last_fullsized":"2024-03-28T01:06:23.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710944+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:06:24.710944+0000","last_peered":"2024-03-28T01:06:24.710944+0000","last_clean":"2024-03-28T01:06:24.710944+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:06:24.710944+0000","last_undegraded":"2024-03-28T01:06:24.710944+0000","last_fullsized":"2024-03-28T01:06:24.710944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723935+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:06:23.723935+0000","last_peered":"2024-03-28T01:06:23.723935+0000","last_clean":"2024-03-28T01:06:23.723935+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:06:23.723935+0000","last_undegraded":"2024-03-28T01:06:23.723935+0000","last_fullsized":"2024-03-28T01:06:23.723935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711041+0000","last_change":"2024-03-28T01:06:21.715183+0000","last_active":"2024-03-28T01:06:24.711041+0000","last_peered":"2024-03-28T01:06:24.711041+0000","last_clean":"2024-03-28T01:06:24.711041+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:06:24.711041+0000","last_undegraded":"2024-03-28T01:06:24.711041+0000","last_fullsized":"2024-03-28T01:06:24.711041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724403+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:06:23.724403+0000","last_peered":"2024-03-28T01:06:23.724403+0000","last_clean":"2024-03-28T01:06:23.724403+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:06:23.724403+0000","last_undegraded":"2024-03-28T01:06:23.724403+0000","last_fullsized":"2024-03-28T01:06:23.724403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746277+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:06:22.746277+0000","last_peered":"2024-03-28T01:06:22.746277+0000","last_clean":"2024-03-28T01:06:22.746277+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:06:22.746277+0000","last_undegraded":"2024-03-28T01:06:22.746277+0000","last_fullsized":"2024-03-28T01:06:22.746277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717267+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:06:24.717267+0000","last_peered":"2024-03-28T01:06:24.717267+0000","last_clean":"2024-03-28T01:06:24.717267+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:06:24.717267+0000","last_undegraded":"2024-03-28T01:06:24.717267+0000","last_fullsized":"2024-03-28T01:06:24.717267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746087+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:06:22.746087+0000","last_peered":"2024-03-28T01:06:22.746087+0000","last_clean":"2024-03-28T01:06:22.746087+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:06:22.746087+0000","last_undegraded":"2024-03-28T01:06:22.746087+0000","last_fullsized":"2024-03-28T01:06:22.746087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:09:36.130037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.748519+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:06:22.748519+0000","last_peered":"2024-03-28T01:06:22.748519+0000","last_clean":"2024-03-28T01:06:22.748519+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:06:22.748519+0000","last_undegraded":"2024-03-28T01:06:22.748519+0000","last_fullsized":"2024-03-28T01:06:22.748519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.735623+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:06:22.735623+0000","last_peered":"2024-03-28T01:06:22.735623+0000","last_clean":"2024-03-28T01:06:22.735623+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:06:22.735623+0000","last_undegraded":"2024-03-28T01:06:22.735623+0000","last_fullsized":"2024-03-28T01:06:22.735623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.745329+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:06:22.745329+0000","last_peered":"2024-03-28T01:06:22.745329+0000","last_clean":"2024-03-28T01:06:22.745329+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:06:22.745329+0000","last_undegraded":"2024-03-28T01:06:22.745329+0000","last_fullsized":"2024-03-28T01:06:22.745329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723980+0000","last_change":"2024-03-28T01:06:22.742813+0000","last_active":"2024-03-28T01:06:23.723980+0000","last_peered":"2024-03-28T01:06:23.723980+0000","last_clean":"2024-03-28T01:06:23.723980+0000","last_became_active":"2024-03-28T01:06:22.736152+0000","last_became_peered":"2024-03-28T01:06:22.736152+0000","last_unstale":"2024-03-28T01:06:23.723980+0000","last_undegraded":"2024-03-28T01:06:23.723980+0000","last_fullsized":"2024-03-28T01:06:23.723980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725147+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:06:23.725147+0000","last_peered":"2024-03-28T01:06:23.725147+0000","last_clean":"2024-03-28T01:06:23.725147+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:06:23.725147+0000","last_undegraded":"2024-03-28T01:06:23.725147+0000","last_fullsized":"2024-03-28T01:06:23.725147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:32.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725002+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:06:23.725002+0000","last_peered":"2024-03-28T01:06:23.725002+0000","last_clean":"2024-03-28T01:06:23.725002+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:06:23.725002+0000","last_undegraded":"2024-03-28T01:06:23.725002+0000","last_fullsized":"2024-03-28T01:06:23.725002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724107+0000","last_change":"2024-03-28T01:06:21.727858+0000","last_active":"2024-03-28T01:06:23.724107+0000","last_peered":"2024-03-28T01:06:23.724107+0000","last_clean":"2024-03-28T01:06:23.724107+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:06:23.724107+0000","last_undegraded":"2024-03-28T01:06:23.724107+0000","last_fullsized":"2024-03-28T01:06:23.724107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:15:55.049218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:06:27.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:06:27.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.566+0000 7f7cf7200640 1 Processor -- start 2024-03-28T01:06:27.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.566+0000 7f7cf7200640 1 -- start start 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.569+0000 7f7cf7200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7cf00b8d00 0x7f7cf014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.569+0000 7f7cf7200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf00b55b0 con 0x7f7cf00b51d0 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.569+0000 7f7cf7200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf00b9190 con 0x7f7cf00b6b80 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.569+0000 7f7cf7200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf014a820 con 0x7f7cf00b8d00 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.569+0000 7f7cf5e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7cf00b51d0 0x7f7cf00b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41018/0 (socket says 172.21.15.33:41018) 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.569+0000 7f7cf5e00640 1 -- 172.21.15.33:0/2991946677 learned_addr learned my addr 172.21.15.33:0/2991946677 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.569+0000 7f7cf5400640 1 --2- 172.21.15.33:0/2991946677 >> v2:172.21.15.194:3300/0 conn(0x7f7cf00b8d00 0x7f7cf014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165859345 0 0) 0x7f7cf00b55b0 con 0x7f7cf00b51d0 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cd8003520 con 0x7f7cf00b51d0 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1199524292 0 0) 0x7f7cd8003520 con 0x7f7cf00b51d0 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 >> v2:172.21.15.194:3300/0 conn(0x7f7cf00b8d00 msgr2=0x7f7cf014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 --2- 172.21.15.33:0/2991946677 >> v2:172.21.15.194:3300/0 conn(0x7f7cf00b8d00 0x7f7cf014ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 >> v1:172.21.15.194:6789/0 conn(0x7f7cf00b6b80 legacy=0x7f7cf014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf00b9800 con 0x7f7cf00b51d0 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f7ce4001670 con 0x7f7cf00b51d0 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 >> v1:172.21.15.33:6789/0 conn(0x7f7cf00b51d0 legacy=0x7f7cf00b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd80039b0 0x7f7cd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cd80051c0 0x7f7cd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd800c030 con 0x7f7cd8004800 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd800c1e0 con 0x7f7cd80039b0 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd800c3e0 con 0x7f7cd80051c0 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.570+0000 7f7cf5400640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.571+0000 7f7cf5e00640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd80039b0 0x7f7cd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.571+0000 7f7cf5400640 1 -- 172.21.15.33:0/2991946677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cd80051c0 msgr2=0x7f7cd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.571+0000 7f7cf5400640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cd80051c0 0x7f7cd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.571+0000 7f7cf5400640 1 -- 172.21.15.33:0/2991946677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd80039b0 msgr2=0x7f7cd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.571+0000 7f7cf5400640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd80039b0 0x7f7cd8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.571+0000 7f7cf5400640 1 -- 172.21.15.33:0/2991946677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd8003520 con 0x7f7cd8004800 2024-03-28T01:06:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.571+0000 7f7cf6800640 1 -- 172.21.15.33:0/2991946677 reap_dead start 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.572+0000 7f7cf5400640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cd8004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009ac0 tx=0x7f7cd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3a09169170955d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.572+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0039070 con 0x7f7cd8004800 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.572+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce002c710 con 0x7f7cd8004800 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.572+0000 7f7cf4a00640 1 -- 172.21.15.33:0/2991946677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0009690 con 0x7f7cd8004800 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.573+0000 7f7cf7200640 1 -- 172.21.15.33:0/2991946677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 msgr2=0x7f7cd8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.573+0000 7f7cf7200640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cd8004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009ac0 tx=0x7f7cd8005830 comp rx=0 tx=0).stop 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.573+0000 7f7cf7200640 1 -- 172.21.15.33:0/2991946677 shutdown_connections 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.573+0000 7f7cf7200640 1 --2- 172.21.15.33:0/2991946677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cd8004c80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.573+0000 7f7cf7200640 1 -- 172.21.15.33:0/2991946677 >> 172.21.15.33:0/2991946677 conn(0x7f7cf0072940 msgr2=0x7f7cf0072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.573+0000 7f7cf7200640 1 -- 172.21.15.33:0/2991946677 shutdown_connections 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.573+0000 7f7cf7200640 1 -- 172.21.15.33:0/2991946677 wait complete. 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.575+0000 7f7cf7200640 1 Processor -- start 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.575+0000 7f7cf7200640 1 -- start start 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cf013b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf7200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cf013bbf0 0x7f7cf014a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cf01dce50 0x7f7cf0139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf7200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00b3db0 con 0x7f7cd8004800 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00b39c0 con 0x7f7cf01dce50 2024-03-28T01:06:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf7200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00b3f30 con 0x7f7cf013bbf0 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cf013b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cf013b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36256/0 (socket says 172.21.15.33:36256) 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 -- 172.21.15.33:0/3432325731 learned_addr learned my addr 172.21.15.33:0/3432325731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5400640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cf013bbf0 0x7f7cf014a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 -- 172.21.15.33:0/3432325731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cf013bbf0 msgr2=0x7f7cf014a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cf013bbf0 0x7f7cf014a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 -- 172.21.15.33:0/3432325731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cf01dce50 msgr2=0x7f7cf0139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cf01dce50 0x7f7cf0139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.576+0000 7f7cf5e00640 1 -- 172.21.15.33:0/3432325731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf013c140 con 0x7f7cd8004800 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.577+0000 7f7cf5e00640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cf013b080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400e770 tx=0x7f7ce400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.577+0000 7f7cea000640 1 -- 172.21.15.33:0/3432325731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400ee80 con 0x7f7cd8004800 2024-03-28T01:06:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.577+0000 7f7cea000640 1 -- 172.21.15.33:0/3432325731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4018070 con 0x7f7cd8004800 2024-03-28T01:06:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.577+0000 7f7cea000640 1 -- 172.21.15.33:0/3432325731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400cab0 con 0x7f7cd8004800 2024-03-28T01:06:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.577+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf0144a20 con 0x7f7cd8004800 2024-03-28T01:06:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.577+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf0143a40 con 0x7f7cd8004800 2024-03-28T01:06:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.578+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf0075310 con 0x7f7cd8004800 2024-03-28T01:06:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.579+0000 7f7cea000640 1 -- 172.21.15.33:0/3432325731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7ce400cc50 con 0x7f7cd8004800 2024-03-28T01:06:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.581+0000 7f7cea000640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7cd406bbd0 0x7f7cd406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.581+0000 7f7cea000640 1 -- 172.21.15.33:0/3432325731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ce408a820 con 0x7f7cd8004800 2024-03-28T01:06:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.581+0000 7f7cf5400640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7cd406bbd0 0x7f7cd406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:27.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.582+0000 7f7cf5400640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7cd406bbd0 0x7f7cd406e090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0036000 tx=0x7f7ce0030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:27.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.586+0000 7f7cea000640 1 -- 172.21.15.33:0/3432325731 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f7ce40593a0 con 0x7f7cd8004800 2024-03-28T01:06:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.784+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7cf0146ac0 con 0x7f7cd406bbd0 2024-03-28T01:06:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.794+0000 7f7cea000640 1 -- 172.21.15.33:0/3432325731 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251920 (secure 0 0 0) 0x7f7cf0146ac0 con 0x7f7cd406bbd0 2024-03-28T01:06:27.800 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:06:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7cd406bbd0 msgr2=0x7f7cd406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7cd406bbd0 0x7f7cd406e090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0036000 tx=0x7f7ce0030000 comp rx=0 tx=0).stop 2024-03-28T01:06:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 msgr2=0x7f7cf013b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cf013b080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400e770 tx=0x7f7ce400ec30 comp rx=0 tx=0).stop 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 shutdown_connections 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7cd406bbd0 0x7f7cd406e090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cf01dce50 0x7f7cf0139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7cf013bbf0 0x7f7cf014a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 --2- 172.21.15.33:0/3432325731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd8004800 0x7f7cf013b080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 >> 172.21.15.33:0/3432325731 conn(0x7f7cf0072940 msgr2=0x7f7cf00bd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 shutdown_connections 2024-03-28T01:06:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:27.800+0000 7f7cf7200640 1 -- 172.21.15.33:0/3432325731 wait complete. 2024-03-28T01:06:27.821 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-28T01:06:26.385140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.711491"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750987+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:06:22.750987+0000","last_peered":"2024-03-28T01:06:22.750987+0000","last_clean":"2024-03-28T01:06:22.750987+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:06:22.750987+0000","last_undegraded":"2024-03-28T01:06:22.750987+0000","last_fullsized":"2024-03-28T01:06:22.750987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711746+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:06:24.711746+0000","last_peered":"2024-03-28T01:06:24.711746+0000","last_clean":"2024-03-28T01:06:24.711746+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:06:24.711746+0000","last_undegraded":"2024-03-28T01:06:24.711746+0000","last_fullsized":"2024-03-28T01:06:24.711746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723513+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:06:23.723513+0000","last_peered":"2024-03-28T01:06:23.723513+0000","last_clean":"2024-03-28T01:06:23.723513+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:06:23.723513+0000","last_undegraded":"2024-03-28T01:06:23.723513+0000","last_fullsized":"2024-03-28T01:06:23.723513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.754630+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:06:22.754630+0000","last_peered":"2024-03-28T01:06:22.754630+0000","last_clean":"2024-03-28T01:06:22.754630+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:06:22.754630+0000","last_undegraded":"2024-03-28T01:06:22.754630+0000","last_fullsized":"2024-03-28T01:06:22.754630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750828+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:06:22.750828+0000","last_peered":"2024-03-28T01:06:22.750828+0000","last_clean":"2024-03-28T01:06:22.750828+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:06:22.750828+0000","last_undegraded":"2024-03-28T01:06:22.750828+0000","last_fullsized":"2024-03-28T01:06:22.750828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.744777+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:06:22.744777+0000","last_peered":"2024-03-28T01:06:22.744777+0000","last_clean":"2024-03-28T01:06:22.744777+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:06:22.744777+0000","last_undegraded":"2024-03-28T01:06:22.744777+0000","last_fullsized":"2024-03-28T01:06:22.744777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.749623+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:06:22.749623+0000","last_peered":"2024-03-28T01:06:22.749623+0000","last_clean":"2024-03-28T01:06:22.749623+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:06:22.749623+0000","last_undegraded":"2024-03-28T01:06:22.749623+0000","last_fullsized":"2024-03-28T01:06:22.749623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723763+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:06:23.723763+0000","last_peered":"2024-03-28T01:06:23.723763+0000","last_clean":"2024-03-28T01:06:23.723763+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:06:23.723763+0000","last_undegraded":"2024-03-28T01:06:23.723763+0000","last_fullsized":"2024-03-28T01:06:23.723763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750967+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:06:22.750967+0000","last_peered":"2024-03-28T01:06:22.750967+0000","last_clean":"2024-03-28T01:06:22.750967+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:06:22.750967+0000","last_undegraded":"2024-03-28T01:06:22.750967+0000","last_fullsized":"2024-03-28T01:06:22.750967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718027+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:06:24.718027+0000","last_peered":"2024-03-28T01:06:24.718027+0000","last_clean":"2024-03-28T01:06:24.718027+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:06:24.718027+0000","last_undegraded":"2024-03-28T01:06:24.718027+0000","last_fullsized":"2024-03-28T01:06:24.718027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717838+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:06:24.717838+0000","last_peered":"2024-03-28T01:06:24.717838+0000","last_clean":"2024-03-28T01:06:24.717838+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:06:24.717838+0000","last_undegraded":"2024-03-28T01:06:24.717838+0000","last_fullsized":"2024-03-28T01:06:24.717838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727289+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:06:23.727289+0000","last_peered":"2024-03-28T01:06:23.727289+0000","last_clean":"2024-03-28T01:06:23.727289+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:06:23.727289+0000","last_undegraded":"2024-03-28T01:06:23.727289+0000","last_fullsized":"2024-03-28T01:06:23.727289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724224+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:06:23.724224+0000","last_peered":"2024-03-28T01:06:23.724224+0000","last_clean":"2024-03-28T01:06:23.724224+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:06:23.724224+0000","last_undegraded":"2024-03-28T01:06:23.724224+0000","last_fullsized":"2024-03-28T01:06:23.724224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722772+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:06:23.722772+0000","last_peered":"2024-03-28T01:06:23.722772+0000","last_clean":"2024-03-28T01:06:23.722772+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:06:23.722772+0000","last_undegraded":"2024-03-28T01:06:23.722772+0000","last_fullsized":"2024-03-28T01:06:23.722772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710748+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:06:24.710748+0000","last_peered":"2024-03-28T01:06:24.710748+0000","last_clean":"2024-03-28T01:06:24.710748+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:06:24.710748+0000","last_undegraded":"2024-03-28T01:06:24.710748+0000","last_fullsized":"2024-03-28T01:06:24.710748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.744683+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:06:22.744683+0000","last_peered":"2024-03-28T01:06:22.744683+0000","last_clean":"2024-03-28T01:06:22.744683+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:06:22.744683+0000","last_undegraded":"2024-03-28T01:06:22.744683+0000","last_fullsized":"2024-03-28T01:06:22.744683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752236+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:06:22.752236+0000","last_peered":"2024-03-28T01:06:22.752236+0000","last_clean":"2024-03-28T01:06:22.752236+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:06:22.752236+0000","last_undegraded":"2024-03-28T01:06:22.752236+0000","last_fullsized":"2024-03-28T01:06:22.752236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.742233+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:06:22.742233+0000","last_peered":"2024-03-28T01:06:22.742233+0000","last_clean":"2024-03-28T01:06:22.742233+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:06:22.742233+0000","last_undegraded":"2024-03-28T01:06:22.742233+0000","last_fullsized":"2024-03-28T01:06:22.742233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746843+0000","last_change":"2024-03-28T01:06:22.743819+0000","last_active":"2024-03-28T01:06:22.746843+0000","last_peered":"2024-03-28T01:06:22.746843+0000","last_clean":"2024-03-28T01:06:22.746843+0000","last_became_active":"2024-03-28T01:06:22.742898+0000","last_became_peered":"2024-03-28T01:06:22.742898+0000","last_unstale":"2024-03-28T01:06:22.746843+0000","last_undegraded":"2024-03-28T01:06:22.746843+0000","last_fullsized":"2024-03-28T01:06:22.746843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717295+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:06:24.717295+0000","last_peered":"2024-03-28T01:06:24.717295+0000","last_clean":"2024-03-28T01:06:24.717295+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:06:24.717295+0000","last_undegraded":"2024-03-28T01:06:24.717295+0000","last_fullsized":"2024-03-28T01:06:24.717295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722929+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:06:23.722929+0000","last_peered":"2024-03-28T01:06:23.722929+0000","last_clean":"2024-03-28T01:06:23.722929+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:06:23.722929+0000","last_undegraded":"2024-03-28T01:06:23.722929+0000","last_fullsized":"2024-03-28T01:06:23.722929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752340+0000","last_change":"2024-03-28T01:06:22.749132+0000","last_active":"2024-03-28T01:06:22.752340+0000","last_peered":"2024-03-28T01:06:22.752340+0000","last_clean":"2024-03-28T01:06:22.752340+0000","last_became_active":"2024-03-28T01:06:22.748012+0000","last_became_peered":"2024-03-28T01:06:22.748012+0000","last_unstale":"2024-03-28T01:06:22.752340+0000","last_undegraded":"2024-03-28T01:06:22.752340+0000","last_fullsized":"2024-03-28T01:06:22.752340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724138+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:06:23.724138+0000","last_peered":"2024-03-28T01:06:23.724138+0000","last_clean":"2024-03-28T01:06:23.724138+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:06:23.724138+0000","last_undegraded":"2024-03-28T01:06:23.724138+0000","last_fullsized":"2024-03-28T01:06:23.724138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752128+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:06:22.752128+0000","last_peered":"2024-03-28T01:06:22.752128+0000","last_clean":"2024-03-28T01:06:22.752128+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:06:22.752128+0000","last_undegraded":"2024-03-28T01:06:22.752128+0000","last_fullsized":"2024-03-28T01:06:22.752128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724983+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:06:23.724983+0000","last_peered":"2024-03-28T01:06:23.724983+0000","last_clean":"2024-03-28T01:06:23.724983+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:06:23.724983+0000","last_undegraded":"2024-03-28T01:06:23.724983+0000","last_fullsized":"2024-03-28T01:06:23.724983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718057+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:06:24.718057+0000","last_peered":"2024-03-28T01:06:24.718057+0000","last_clean":"2024-03-28T01:06:24.718057+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:06:24.718057+0000","last_undegraded":"2024-03-28T01:06:24.718057+0000","last_fullsized":"2024-03-28T01:06:24.718057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711534+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:06:24.711534+0000","last_peered":"2024-03-28T01:06:24.711534+0000","last_clean":"2024-03-28T01:06:24.711534+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:06:24.711534+0000","last_undegraded":"2024-03-28T01:06:24.711534+0000","last_fullsized":"2024-03-28T01:06:24.711534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725154+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:06:23.725154+0000","last_peered":"2024-03-28T01:06:23.725154+0000","last_clean":"2024-03-28T01:06:23.725154+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:06:23.725154+0000","last_undegraded":"2024-03-28T01:06:23.725154+0000","last_fullsized":"2024-03-28T01:06:23.725154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723116+0000","last_change":"2024-03-28T01:06:22.731272+0000","last_active":"2024-03-28T01:06:23.723116+0000","last_peered":"2024-03-28T01:06:23.723116+0000","last_clean":"2024-03-28T01:06:23.723116+0000","last_became_active":"2024-03-28T01:06:22.730261+0000","last_became_peered":"2024-03-28T01:06:22.730261+0000","last_unstale":"2024-03-28T01:06:23.723116+0000","last_undegraded":"2024-03-28T01:06:23.723116+0000","last_fullsized":"2024-03-28T01:06:23.723116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752328+0000","last_change":"2024-03-28T01:06:22.746590+0000","last_active":"2024-03-28T01:06:22.752328+0000","last_peered":"2024-03-28T01:06:22.752328+0000","last_clean":"2024-03-28T01:06:22.752328+0000","last_became_active":"2024-03-28T01:06:22.746299+0000","last_became_peered":"2024-03-28T01:06:22.746299+0000","last_unstale":"2024-03-28T01:06:22.752328+0000","last_undegraded":"2024-03-28T01:06:22.752328+0000","last_fullsized":"2024-03-28T01:06:22.752328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.739711+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:06:22.739711+0000","last_peered":"2024-03-28T01:06:22.739711+0000","last_clean":"2024-03-28T01:06:22.739711+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:06:22.739711+0000","last_undegraded":"2024-03-28T01:06:22.739711+0000","last_fullsized":"2024-03-28T01:06:22.739711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717294+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:06:24.717294+0000","last_peered":"2024-03-28T01:06:24.717294+0000","last_clean":"2024-03-28T01:06:24.717294+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:06:24.717294+0000","last_undegraded":"2024-03-28T01:06:24.717294+0000","last_fullsized":"2024-03-28T01:06:24.717294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726504+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:06:23.726504+0000","last_peered":"2024-03-28T01:06:23.726504+0000","last_clean":"2024-03-28T01:06:23.726504+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:06:23.726504+0000","last_undegraded":"2024-03-28T01:06:23.726504+0000","last_fullsized":"2024-03-28T01:06:23.726504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724518+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:06:23.724518+0000","last_peered":"2024-03-28T01:06:23.724518+0000","last_clean":"2024-03-28T01:06:23.724518+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:06:23.724518+0000","last_undegraded":"2024-03-28T01:06:23.724518+0000","last_fullsized":"2024-03-28T01:06:23.724518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723265+0000","last_change":"2024-03-28T01:06:22.717986+0000","last_active":"2024-03-28T01:06:23.723265+0000","last_peered":"2024-03-28T01:06:23.723265+0000","last_clean":"2024-03-28T01:06:23.723265+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:06:23.723265+0000","last_undegraded":"2024-03-28T01:06:23.723265+0000","last_fullsized":"2024-03-28T01:06:23.723265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717592+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:06:24.717592+0000","last_peered":"2024-03-28T01:06:24.717592+0000","last_clean":"2024-03-28T01:06:24.717592+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:06:24.717592+0000","last_undegraded":"2024-03-28T01:06:24.717592+0000","last_fullsized":"2024-03-28T01:06:24.717592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724278+0000","last_change":"2024-03-28T01:06:21.721278+0000","last_active":"2024-03-28T01:06:23.724278+0000","last_peered":"2024-03-28T01:06:23.724278+0000","last_clean":"2024-03-28T01:06:23.724278+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:06:23.724278+0000","last_undegraded":"2024-03-28T01:06:23.724278+0000","last_fullsized":"2024-03-28T01:06:23.724278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723815+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:06:23.723815+0000","last_peered":"2024-03-28T01:06:23.723815+0000","last_clean":"2024-03-28T01:06:23.723815+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:06:23.723815+0000","last_undegraded":"2024-03-28T01:06:23.723815+0000","last_fullsized":"2024-03-28T01:06:23.723815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.727997+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:06:22.727997+0000","last_peered":"2024-03-28T01:06:22.727997+0000","last_clean":"2024-03-28T01:06:22.727997+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:06:22.727997+0000","last_undegraded":"2024-03-28T01:06:22.727997+0000","last_fullsized":"2024-03-28T01:06:22.727997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710729+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:06:24.710729+0000","last_peered":"2024-03-28T01:06:24.710729+0000","last_clean":"2024-03-28T01:06:24.710729+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:06:24.710729+0000","last_undegraded":"2024-03-28T01:06:24.710729+0000","last_fullsized":"2024-03-28T01:06:24.710729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717704+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:06:24.717704+0000","last_peered":"2024-03-28T01:06:24.717704+0000","last_clean":"2024-03-28T01:06:24.717704+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:06:24.717704+0000","last_undegraded":"2024-03-28T01:06:24.717704+0000","last_fullsized":"2024-03-28T01:06:24.717704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752268+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:06:22.752268+0000","last_peered":"2024-03-28T01:06:22.752268+0000","last_clean":"2024-03-28T01:06:22.752268+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:06:22.752268+0000","last_undegraded":"2024-03-28T01:06:22.752268+0000","last_fullsized":"2024-03-28T01:06:22.752268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.731365+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:06:22.731365+0000","last_peered":"2024-03-28T01:06:22.731365+0000","last_clean":"2024-03-28T01:06:22.731365+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:06:22.731365+0000","last_undegraded":"2024-03-28T01:06:22.731365+0000","last_fullsized":"2024-03-28T01:06:22.731365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724058+0000","last_change":"2024-03-28T01:06:22.743892+0000","last_active":"2024-03-28T01:06:23.724058+0000","last_peered":"2024-03-28T01:06:23.724058+0000","last_clean":"2024-03-28T01:06:23.724058+0000","last_became_active":"2024-03-28T01:06:22.743113+0000","last_became_peered":"2024-03-28T01:06:22.743113+0000","last_unstale":"2024-03-28T01:06:23.724058+0000","last_undegraded":"2024-03-28T01:06:23.724058+0000","last_fullsized":"2024-03-28T01:06:23.724058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711550+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:06:24.711550+0000","last_peered":"2024-03-28T01:06:24.711550+0000","last_clean":"2024-03-28T01:06:24.711550+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:06:24.711550+0000","last_undegraded":"2024-03-28T01:06:24.711550+0000","last_fullsized":"2024-03-28T01:06:24.711550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727568+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:06:23.727568+0000","last_peered":"2024-03-28T01:06:23.727568+0000","last_clean":"2024-03-28T01:06:23.727568+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:06:23.727568+0000","last_undegraded":"2024-03-28T01:06:23.727568+0000","last_fullsized":"2024-03-28T01:06:23.727568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717385+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:06:24.717385+0000","last_peered":"2024-03-28T01:06:24.717385+0000","last_clean":"2024-03-28T01:06:24.717385+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:06:24.717385+0000","last_undegraded":"2024-03-28T01:06:24.717385+0000","last_fullsized":"2024-03-28T01:06:24.717385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746820+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:06:22.746820+0000","last_peered":"2024-03-28T01:06:22.746820+0000","last_clean":"2024-03-28T01:06:22.746820+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:06:22.746820+0000","last_undegraded":"2024-03-28T01:06:22.746820+0000","last_fullsized":"2024-03-28T01:06:22.746820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723346+0000","last_change":"2024-03-28T01:06:21.727305+0000","last_active":"2024-03-28T01:06:23.723346+0000","last_peered":"2024-03-28T01:06:23.723346+0000","last_clean":"2024-03-28T01:06:23.723346+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:06:23.723346+0000","last_undegraded":"2024-03-28T01:06:23.723346+0000","last_fullsized":"2024-03-28T01:06:23.723346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722951+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:06:23.722951+0000","last_peered":"2024-03-28T01:06:23.722951+0000","last_clean":"2024-03-28T01:06:23.722951+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:06:23.722951+0000","last_undegraded":"2024-03-28T01:06:23.722951+0000","last_fullsized":"2024-03-28T01:06:23.722951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.729397+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:06:22.729397+0000","last_peered":"2024-03-28T01:06:22.729397+0000","last_clean":"2024-03-28T01:06:22.729397+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:06:22.729397+0000","last_undegraded":"2024-03-28T01:06:22.729397+0000","last_fullsized":"2024-03-28T01:06:22.729397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746354+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:06:22.746354+0000","last_peered":"2024-03-28T01:06:22.746354+0000","last_clean":"2024-03-28T01:06:22.746354+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:06:22.746354+0000","last_undegraded":"2024-03-28T01:06:22.746354+0000","last_fullsized":"2024-03-28T01:06:22.746354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:09:36.130037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717022+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:06:24.717022+0000","last_peered":"2024-03-28T01:06:24.717022+0000","last_clean":"2024-03-28T01:06:24.717022+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:06:24.717022+0000","last_undegraded":"2024-03-28T01:06:24.717022+0000","last_fullsized":"2024-03-28T01:06:24.717022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717552+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:06:24.717552+0000","last_peered":"2024-03-28T01:06:24.717552+0000","last_clean":"2024-03-28T01:06:24.717552+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:06:24.717552+0000","last_undegraded":"2024-03-28T01:06:24.717552+0000","last_fullsized":"2024-03-28T01:06:24.717552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.738398+0000","last_change":"2024-03-28T01:06:21.719466+0000","last_active":"2024-03-28T01:06:22.738398+0000","last_peered":"2024-03-28T01:06:22.738398+0000","last_clean":"2024-03-28T01:06:22.738398+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:06:22.738398+0000","last_undegraded":"2024-03-28T01:06:22.738398+0000","last_fullsized":"2024-03-28T01:06:22.738398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746930+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:06:22.746930+0000","last_peered":"2024-03-28T01:06:22.746930+0000","last_clean":"2024-03-28T01:06:22.746930+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:06:22.746930+0000","last_undegraded":"2024-03-28T01:06:22.746930+0000","last_fullsized":"2024-03-28T01:06:22.746930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727523+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:06:23.727523+0000","last_peered":"2024-03-28T01:06:23.727523+0000","last_clean":"2024-03-28T01:06:23.727523+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:06:23.727523+0000","last_undegraded":"2024-03-28T01:06:23.727523+0000","last_fullsized":"2024-03-28T01:06:23.727523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.749659+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:06:22.749659+0000","last_peered":"2024-03-28T01:06:22.749659+0000","last_clean":"2024-03-28T01:06:22.749659+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:06:22.749659+0000","last_undegraded":"2024-03-28T01:06:22.749659+0000","last_fullsized":"2024-03-28T01:06:22.749659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.742461+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:06:22.742461+0000","last_peered":"2024-03-28T01:06:22.742461+0000","last_clean":"2024-03-28T01:06:22.742461+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:06:22.742461+0000","last_undegraded":"2024-03-28T01:06:22.742461+0000","last_fullsized":"2024-03-28T01:06:22.742461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.755435+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:06:22.755435+0000","last_peered":"2024-03-28T01:06:22.755435+0000","last_clean":"2024-03-28T01:06:22.755435+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:06:22.755435+0000","last_undegraded":"2024-03-28T01:06:22.755435+0000","last_fullsized":"2024-03-28T01:06:22.755435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711878+0000","last_change":"2024-03-28T01:06:21.718159+0000","last_active":"2024-03-28T01:06:24.711878+0000","last_peered":"2024-03-28T01:06:24.711878+0000","last_clean":"2024-03-28T01:06:24.711878+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:06:24.711878+0000","last_undegraded":"2024-03-28T01:06:24.711878+0000","last_fullsized":"2024-03-28T01:06:24.711878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.754652+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:06:22.754652+0000","last_peered":"2024-03-28T01:06:22.754652+0000","last_clean":"2024-03-28T01:06:22.754652+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:06:22.754652+0000","last_undegraded":"2024-03-28T01:06:22.754652+0000","last_fullsized":"2024-03-28T01:06:22.754652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.720119+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:06:22.720119+0000","last_peered":"2024-03-28T01:06:22.720119+0000","last_clean":"2024-03-28T01:06:22.720119+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:06:22.720119+0000","last_undegraded":"2024-03-28T01:06:22.720119+0000","last_fullsized":"2024-03-28T01:06:22.720119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.750924+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:06:22.750924+0000","last_peered":"2024-03-28T01:06:22.750924+0000","last_clean":"2024-03-28T01:06:22.750924+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:06:22.750924+0000","last_undegraded":"2024-03-28T01:06:22.750924+0000","last_fullsized":"2024-03-28T01:06:22.750924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.721860+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:06:22.721860+0000","last_peered":"2024-03-28T01:06:22.721860+0000","last_clean":"2024-03-28T01:06:22.721860+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:06:22.721860+0000","last_undegraded":"2024-03-28T01:06:22.721860+0000","last_fullsized":"2024-03-28T01:06:22.721860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.729015+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:06:22.729015+0000","last_peered":"2024-03-28T01:06:22.729015+0000","last_clean":"2024-03-28T01:06:22.729015+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:06:22.729015+0000","last_undegraded":"2024-03-28T01:06:22.729015+0000","last_fullsized":"2024-03-28T01:06:22.729015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724610+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:06:23.724610+0000","last_peered":"2024-03-28T01:06:23.724610+0000","last_clean":"2024-03-28T01:06:23.724610+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:06:23.724610+0000","last_undegraded":"2024-03-28T01:06:23.724610+0000","last_fullsized":"2024-03-28T01:06:23.724610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.721801+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:06:22.721801+0000","last_peered":"2024-03-28T01:06:22.721801+0000","last_clean":"2024-03-28T01:06:22.721801+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:06:22.721801+0000","last_undegraded":"2024-03-28T01:06:22.721801+0000","last_fullsized":"2024-03-28T01:06:22.721801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722723+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:06:23.722723+0000","last_peered":"2024-03-28T01:06:23.722723+0000","last_clean":"2024-03-28T01:06:23.722723+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:06:23.722723+0000","last_undegraded":"2024-03-28T01:06:23.722723+0000","last_fullsized":"2024-03-28T01:06:23.722723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.716574+0000","last_change":"2024-03-28T01:06:21.720753+0000","last_active":"2024-03-28T01:06:22.716574+0000","last_peered":"2024-03-28T01:06:22.716574+0000","last_clean":"2024-03-28T01:06:22.716574+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:06:22.716574+0000","last_undegraded":"2024-03-28T01:06:22.716574+0000","last_fullsized":"2024-03-28T01:06:22.716574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710942+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:06:24.710942+0000","last_peered":"2024-03-28T01:06:24.710942+0000","last_clean":"2024-03-28T01:06:24.710942+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:06:24.710942+0000","last_undegraded":"2024-03-28T01:06:24.710942+0000","last_fullsized":"2024-03-28T01:06:24.710942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.729552+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:06:22.729552+0000","last_peered":"2024-03-28T01:06:22.729552+0000","last_clean":"2024-03-28T01:06:22.729552+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:06:22.729552+0000","last_undegraded":"2024-03-28T01:06:22.729552+0000","last_fullsized":"2024-03-28T01:06:22.729552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727032+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:06:23.727032+0000","last_peered":"2024-03-28T01:06:23.727032+0000","last_clean":"2024-03-28T01:06:23.727032+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:06:23.727032+0000","last_undegraded":"2024-03-28T01:06:23.727032+0000","last_fullsized":"2024-03-28T01:06:23.727032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711510+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:06:24.711510+0000","last_peered":"2024-03-28T01:06:24.711510+0000","last_clean":"2024-03-28T01:06:24.711510+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:06:24.711510+0000","last_undegraded":"2024-03-28T01:06:24.711510+0000","last_fullsized":"2024-03-28T01:06:24.711510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.752079+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:06:22.752079+0000","last_peered":"2024-03-28T01:06:22.752079+0000","last_clean":"2024-03-28T01:06:22.752079+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:06:22.752079+0000","last_undegraded":"2024-03-28T01:06:22.752079+0000","last_fullsized":"2024-03-28T01:06:22.752079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.725638+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:06:22.725638+0000","last_peered":"2024-03-28T01:06:22.725638+0000","last_clean":"2024-03-28T01:06:22.725638+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:06:22.725638+0000","last_undegraded":"2024-03-28T01:06:22.725638+0000","last_fullsized":"2024-03-28T01:06:22.725638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711037+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:06:24.711037+0000","last_peered":"2024-03-28T01:06:24.711037+0000","last_clean":"2024-03-28T01:06:24.711037+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:06:24.711037+0000","last_undegraded":"2024-03-28T01:06:24.711037+0000","last_fullsized":"2024-03-28T01:06:24.711037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.715740+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:06:24.715740+0000","last_peered":"2024-03-28T01:06:24.715740+0000","last_clean":"2024-03-28T01:06:24.715740+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:06:24.715740+0000","last_undegraded":"2024-03-28T01:06:24.715740+0000","last_fullsized":"2024-03-28T01:06:24.715740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:34.313143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725201+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:06:23.725201+0000","last_peered":"2024-03-28T01:06:23.725201+0000","last_clean":"2024-03-28T01:06:23.725201+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:06:23.725201+0000","last_undegraded":"2024-03-28T01:06:23.725201+0000","last_fullsized":"2024-03-28T01:06:23.725201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717736+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:06:24.717736+0000","last_peered":"2024-03-28T01:06:24.717736+0000","last_clean":"2024-03-28T01:06:24.717736+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:06:24.717736+0000","last_undegraded":"2024-03-28T01:06:24.717736+0000","last_fullsized":"2024-03-28T01:06:24.717736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723864+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:06:23.723864+0000","last_peered":"2024-03-28T01:06:23.723864+0000","last_clean":"2024-03-28T01:06:23.723864+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:06:23.723864+0000","last_undegraded":"2024-03-28T01:06:23.723864+0000","last_fullsized":"2024-03-28T01:06:23.723864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.721637+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:06:22.721637+0000","last_peered":"2024-03-28T01:06:22.721637+0000","last_clean":"2024-03-28T01:06:22.721637+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:06:22.721637+0000","last_undegraded":"2024-03-28T01:06:22.721637+0000","last_fullsized":"2024-03-28T01:06:22.721637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726343+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:06:23.726343+0000","last_peered":"2024-03-28T01:06:23.726343+0000","last_clean":"2024-03-28T01:06:23.726343+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:06:23.726343+0000","last_undegraded":"2024-03-28T01:06:23.726343+0000","last_fullsized":"2024-03-28T01:06:23.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710944+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:06:24.710944+0000","last_peered":"2024-03-28T01:06:24.710944+0000","last_clean":"2024-03-28T01:06:24.710944+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:06:24.710944+0000","last_undegraded":"2024-03-28T01:06:24.710944+0000","last_fullsized":"2024-03-28T01:06:24.710944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723935+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:06:23.723935+0000","last_peered":"2024-03-28T01:06:23.723935+0000","last_clean":"2024-03-28T01:06:23.723935+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:06:23.723935+0000","last_undegraded":"2024-03-28T01:06:23.723935+0000","last_fullsized":"2024-03-28T01:06:23.723935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711041+0000","last_change":"2024-03-28T01:06:21.715183+0000","last_active":"2024-03-28T01:06:24.711041+0000","last_peered":"2024-03-28T01:06:24.711041+0000","last_clean":"2024-03-28T01:06:24.711041+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:06:24.711041+0000","last_undegraded":"2024-03-28T01:06:24.711041+0000","last_fullsized":"2024-03-28T01:06:24.711041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724403+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:06:23.724403+0000","last_peered":"2024-03-28T01:06:23.724403+0000","last_clean":"2024-03-28T01:06:23.724403+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:06:23.724403+0000","last_undegraded":"2024-03-28T01:06:23.724403+0000","last_fullsized":"2024-03-28T01:06:23.724403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746277+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:06:22.746277+0000","last_peered":"2024-03-28T01:06:22.746277+0000","last_clean":"2024-03-28T01:06:22.746277+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:06:22.746277+0000","last_undegraded":"2024-03-28T01:06:22.746277+0000","last_fullsized":"2024-03-28T01:06:22.746277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717267+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:06:24.717267+0000","last_peered":"2024-03-28T01:06:24.717267+0000","last_clean":"2024-03-28T01:06:24.717267+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:06:24.717267+0000","last_undegraded":"2024-03-28T01:06:24.717267+0000","last_fullsized":"2024-03-28T01:06:24.717267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.746087+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:06:22.746087+0000","last_peered":"2024-03-28T01:06:22.746087+0000","last_clean":"2024-03-28T01:06:22.746087+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:06:22.746087+0000","last_undegraded":"2024-03-28T01:06:22.746087+0000","last_fullsized":"2024-03-28T01:06:22.746087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:09:36.130037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.748519+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:06:22.748519+0000","last_peered":"2024-03-28T01:06:22.748519+0000","last_clean":"2024-03-28T01:06:22.748519+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:06:22.748519+0000","last_undegraded":"2024-03-28T01:06:22.748519+0000","last_fullsized":"2024-03-28T01:06:22.748519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.735623+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:06:22.735623+0000","last_peered":"2024-03-28T01:06:22.735623+0000","last_clean":"2024-03-28T01:06:22.735623+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:06:22.735623+0000","last_undegraded":"2024-03-28T01:06:22.735623+0000","last_fullsized":"2024-03-28T01:06:22.735623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-28T01:06:22.745329+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:06:22.745329+0000","last_peered":"2024-03-28T01:06:22.745329+0000","last_clean":"2024-03-28T01:06:22.745329+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:06:22.745329+0000","last_undegraded":"2024-03-28T01:06:22.745329+0000","last_fullsized":"2024-03-28T01:06:22.745329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723980+0000","last_change":"2024-03-28T01:06:22.742813+0000","last_active":"2024-03-28T01:06:23.723980+0000","last_peered":"2024-03-28T01:06:23.723980+0000","last_clean":"2024-03-28T01:06:23.723980+0000","last_became_active":"2024-03-28T01:06:22.736152+0000","last_became_peered":"2024-03-28T01:06:22.736152+0000","last_unstale":"2024-03-28T01:06:23.723980+0000","last_undegraded":"2024-03-28T01:06:23.723980+0000","last_fullsized":"2024-03-28T01:06:23.723980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725147+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:06:23.725147+0000","last_peered":"2024-03-28T01:06:23.725147+0000","last_clean":"2024-03-28T01:06:23.725147+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:06:23.725147+0000","last_undegraded":"2024-03-28T01:06:23.725147+0000","last_fullsized":"2024-03-28T01:06:23.725147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:32.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725002+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:06:23.725002+0000","last_peered":"2024-03-28T01:06:23.725002+0000","last_clean":"2024-03-28T01:06:23.725002+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:06:23.725002+0000","last_undegraded":"2024-03-28T01:06:23.725002+0000","last_fullsized":"2024-03-28T01:06:23.725002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724107+0000","last_change":"2024-03-28T01:06:21.727858+0000","last_active":"2024-03-28T01:06:23.724107+0000","last_peered":"2024-03-28T01:06:23.724107+0000","last_clean":"2024-03-28T01:06:23.724107+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:06:23.724107+0000","last_undegraded":"2024-03-28T01:06:23.724107+0000","last_fullsized":"2024-03-28T01:06:23.724107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:15:55.049218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:06:27.825 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T01:06:27.825 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-28T01:06:27.825 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-28T01:06:27.826 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-28T01:06:27.826 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-28T01:06:28.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.074+0000 7f6a09a00640 1 Processor -- start 2024-03-28T01:06:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.074+0000 7f6a09a00640 1 -- start start 2024-03-28T01:06:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.075+0000 7f6a09a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a040b8d00 0x7f6a0414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.075+0000 7f6a09a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b6b80 0x7f6a0414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.075+0000 7f6a09a00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a040b55b0 con 0x7f6a040b51d0 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.075+0000 7f6a09a00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0414a820 con 0x7f6a040b8d00 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.075+0000 7f6a09a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b9190 con 0x7f6a040b6b80 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f6a03e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6a040b51d0 0x7f6a040b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41028/0 (socket says 172.21.15.33:41028) 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f6a03e00640 1 -- 172.21.15.33:0/393255568 learned_addr learned my addr 172.21.15.33:0/393255568 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f69fb400640 1 --2- 172.21.15.33:0/393255568 >> v2:172.21.15.33:3300/0 conn(0x7f6a040b8d00 0x7f6a0414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f6a09000640 1 --2- 172.21.15.33:0/393255568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b6b80 0x7f6a0414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f69fb400640 1 -- 172.21.15.33:0/393255568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b6b80 msgr2=0x7f6a0414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f69fb400640 1 --2- 172.21.15.33:0/393255568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b6b80 0x7f6a0414b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f69fb400640 1 -- 172.21.15.33:0/393255568 >> v1:172.21.15.33:6789/0 conn(0x7f6a040b51d0 legacy=0x7f6a040b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f69fb400640 1 -- 172.21.15.33:0/393255568 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a040b9800 con 0x7f6a040b8d00 2024-03-28T01:06:28.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.076+0000 7f6a03400640 1 -- 172.21.15.33:0/393255568 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652230874 0 0) 0x7f6a040b55b0 con 0x7f6a040b51d0 2024-03-28T01:06:28.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.078+0000 7f69fb400640 1 --2- 172.21.15.33:0/393255568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a040b8d00 0x7f6a0414ac70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b19a2b9f3af9bcf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:28.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.078+0000 7f6a03400640 1 -- 172.21.15.33:0/393255568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4035020 con 0x7f6a040b8d00 2024-03-28T01:06:28.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.078+0000 7f6a03400640 1 -- 172.21.15.33:0/393255568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4031810 con 0x7f6a040b8d00 2024-03-28T01:06:28.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.078+0000 7f6a03400640 1 -- 172.21.15.33:0/393255568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4031be0 con 0x7f6a040b8d00 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 -- 172.21.15.33:0/393255568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a040b8d00 msgr2=0x7f6a0414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 --2- 172.21.15.33:0/393255568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a040b8d00 0x7f6a0414ac70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402b8b0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 -- 172.21.15.33:0/393255568 shutdown_connections 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 --2- 172.21.15.33:0/393255568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b6b80 0x7f6a0414b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 --2- 172.21.15.33:0/393255568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a040b8d00 0x7f6a0414ac70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 -- 172.21.15.33:0/393255568 >> 172.21.15.33:0/393255568 conn(0x7f6a04072940 msgr2=0x7f6a04072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 -- 172.21.15.33:0/393255568 shutdown_connections 2024-03-28T01:06:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.079+0000 7f6a09a00640 1 -- 172.21.15.33:0/393255568 wait complete. 2024-03-28T01:06:28.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.082+0000 7f6a09a00640 1 Processor -- start 2024-03-28T01:06:28.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.082+0000 7f6a09a00640 1 -- start start 2024-03-28T01:06:28.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.083+0000 7f6a09a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b51d0 0x7f6a0413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.083+0000 7f6a09a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a040b6b80 0x7f6a041dd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.083+0000 7f6a09a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a041ddb40 0x7f6a04138c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.083+0000 7f6a09a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b44c0 con 0x7f6a041ddb40 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.083+0000 7f6a09a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0414b6f0 con 0x7f6a040b6b80 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.083+0000 7f6a09a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b39c0 con 0x7f6a040b51d0 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a041ddb40 0x7f6a04138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a041ddb40 0x7f6a04138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36280/0 (socket says 172.21.15.33:36280) 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 -- 172.21.15.33:0/1352912272 learned_addr learned my addr 172.21.15.33:0/1352912272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a03e00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b51d0 0x7f6a0413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f69fb400640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a040b6b80 0x7f6a041dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 -- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b51d0 msgr2=0x7f6a0413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b51d0 0x7f6a0413ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 -- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a040b6b80 msgr2=0x7f6a041dd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a040b6b80 0x7f6a041dd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 -- 172.21.15.33:0/1352912272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0413b190 con 0x7f6a041ddb40 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a03e00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b51d0 0x7f6a0413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f69fb400640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a040b6b80 0x7f6a041dd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.084+0000 7f6a09000640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a041ddb40 0x7f6a04138c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69fc007e60 tx=0x7f69fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.085+0000 7f6a00c00640 1 -- 172.21.15.33:0/1352912272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc014620 con 0x7f6a041ddb40 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.085+0000 7f6a00c00640 1 -- 172.21.15.33:0/1352912272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc019070 con 0x7f6a041ddb40 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.085+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a040bacb0 con 0x7f6a041ddb40 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.085+0000 7f6a00c00640 1 -- 172.21.15.33:0/1352912272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc014a00 con 0x7f6a041ddb40 2024-03-28T01:06:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.085+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a041426a0 con 0x7f6a041ddb40 2024-03-28T01:06:28.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.086+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a040b7000 con 0x7f6a041ddb40 2024-03-28T01:06:28.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.087+0000 7f6a00c00640 1 -- 172.21.15.33:0/1352912272 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f69fc014ba0 con 0x7f6a041ddb40 2024-03-28T01:06:28.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.088+0000 7f6a00c00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f69e006bbd0 0x7f69e006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.089+0000 7f6a00c00640 1 -- 172.21.15.33:0/1352912272 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69fc08b2f0 con 0x7f6a041ddb40 2024-03-28T01:06:28.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.089+0000 7f6a03e00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f69e006bbd0 0x7f69e006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.089+0000 7f6a03e00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f69e006bbd0 0x7f69e006e090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69ec000f60 tx=0x7f69ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:28.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.092+0000 7f6a00c00640 1 -- 172.21.15.33:0/1352912272 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f6a040b7000 con 0x7f6a041ddb40 2024-03-28T01:06:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.406+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6a04075310 con 0x7f6a041ddb40 2024-03-28T01:06:28.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.411+0000 7f6a00c00640 1 -- 172.21.15.33:0/1352912272 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+172575 (secure 0 0 0) 0x7f69fc059e70 con 0x7f6a041ddb40 2024-03-28T01:06:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f69e006bbd0 msgr2=0x7f69e006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f69e006bbd0 0x7f69e006e090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69ec000f60 tx=0x7f69ec009040 comp rx=0 tx=0).stop 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a041ddb40 msgr2=0x7f6a04138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a041ddb40 0x7f6a04138c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69fc007e60 tx=0x7f69fc00e4b0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 shutdown_connections 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f69e006bbd0 0x7f69e006e090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a041ddb40 0x7f6a04138c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a040b6b80 0x7f6a041dd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 --2- 172.21.15.33:0/1352912272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a040b51d0 0x7f6a0413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 >> 172.21.15.33:0/1352912272 conn(0x7f6a04072940 msgr2=0x7f6a04146180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 shutdown_connections 2024-03-28T01:06:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.417+0000 7f6a09a00640 1 -- 172.21.15.33:0/1352912272 wait complete. 2024-03-28T01:06:28.436 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":199212674},{"type":"v1","addr":"172.21.15.33:6833","nonce":199212674}]},"active_addr":"172.21.15.33:6833/199212674","active_change":"2024-03-28T01:06:02.364879+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3517949825}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1751683649}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":511698199}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1712728653}]}]} 2024-03-28T01:06:28.440 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-28T01:06:28.440 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-28T01:06:28.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.641+0000 7f5297e00640 1 Processor -- start 2024-03-28T01:06:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.641+0000 7f5297e00640 1 -- start start 2024-03-28T01:06:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.642+0000 7f5297e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f529813f500 0x7f52981485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.642+0000 7f5297e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529814f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.642+0000 7f5297e00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52980b2510 con 0x7f52980b2130 2024-03-28T01:06:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.642+0000 7f5297e00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5298148f90 con 0x7f529813f500 2024-03-28T01:06:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.642+0000 7f5297e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980b4670 con 0x7f5298148b20 2024-03-28T01:06:28.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.643+0000 7f5296000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f529813f500 0x7f52981485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.643+0000 7f5296000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f529813f500 0x7f52981485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44254/0 (socket says 172.21.15.33:44254) 2024-03-28T01:06:28.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.643+0000 7f5296000640 1 -- 172.21.15.33:0/2268203386 learned_addr learned my addr 172.21.15.33:0/2268203386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.643+0000 7f5297400640 1 --2- 172.21.15.33:0/2268203386 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.644+0000 7f5297400640 1 -- 172.21.15.33:0/2268203386 >> v2:172.21.15.194:3300/0 conn(0x7f529813f500 msgr2=0x7f52981485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.644+0000 7f5297400640 1 --2- 172.21.15.33:0/2268203386 >> v2:172.21.15.194:3300/0 conn(0x7f529813f500 0x7f52981485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.644+0000 7f5297400640 1 -- 172.21.15.33:0/2268203386 >> v1:172.21.15.194:6789/0 conn(0x7f52980b2130 legacy=0x7f529813edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.644+0000 7f5297400640 1 -- 172.21.15.33:0/2268203386 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529806bef0 con 0x7f5298148b20 2024-03-28T01:06:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.645+0000 7f5297400640 1 --2- 172.21.15.33:0/2268203386 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529814f850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f52900097a0 tx=0x7f529002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d914aa8953411768 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:28.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5295600640 1 -- 172.21.15.33:0/2268203386 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290038070 con 0x7f5298148b20 2024-03-28T01:06:28.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5295600640 1 -- 172.21.15.33:0/2268203386 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290031700 con 0x7f5298148b20 2024-03-28T01:06:28.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5295600640 1 -- 172.21.15.33:0/2268203386 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290031ad0 con 0x7f5298148b20 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 -- 172.21.15.33:0/2268203386 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 msgr2=0x7f529814f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 --2- 172.21.15.33:0/2268203386 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529814f850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f52900097a0 tx=0x7f529002b7a0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 -- 172.21.15.33:0/2268203386 shutdown_connections 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 --2- 172.21.15.33:0/2268203386 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529814f850 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 --2- 172.21.15.33:0/2268203386 >> v2:172.21.15.194:3300/0 conn(0x7f529813f500 0x7f52981485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 -- 172.21.15.33:0/2268203386 >> 172.21.15.33:0/2268203386 conn(0x7f52980b89c0 msgr2=0x7f52980b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 -- 172.21.15.33:0/2268203386 shutdown_connections 2024-03-28T01:06:28.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.646+0000 7f5297e00640 1 -- 172.21.15.33:0/2268203386 wait complete. 2024-03-28T01:06:28.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.648+0000 7f5297e00640 1 Processor -- start 2024-03-28T01:06:28.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.648+0000 7f5297e00640 1 -- start start 2024-03-28T01:06:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5297e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52980b2130 0x7f5298140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5297e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529813f500 0x7f52981423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5297e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529813a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5297e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980b4ec0 con 0x7f52980b2130 2024-03-28T01:06:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5297e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529806b6d0 con 0x7f529813f500 2024-03-28T01:06:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5297e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5298072d20 con 0x7f5298148b20 2024-03-28T01:06:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52980b2130 0x7f5298140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52980b2130 0x7f5298140770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36286/0 (socket says 172.21.15.33:36286) 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296a00640 1 -- 172.21.15.33:0/3755440784 learned_addr learned my addr 172.21.15.33:0/3755440784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5297400640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296000640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529813f500 0x7f52981423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296a00640 1 -- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 msgr2=0x7f529813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296a00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296a00640 1 -- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529813f500 msgr2=0x7f52981423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.649+0000 7f5296a00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529813f500 0x7f52981423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f5296a00640 1 -- 172.21.15.33:0/3755440784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52981495f0 con 0x7f52980b2130 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f5297400640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f5296a00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52980b2130 0x7f5298140770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f528800c730 tx=0x7f528800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f528f400640 1 -- 172.21.15.33:0/3755440784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288003b70 con 0x7f52980b2130 2024-03-28T01:06:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f528f400640 1 -- 172.21.15.33:0/3755440784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288003d30 con 0x7f52980b2130 2024-03-28T01:06:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52981498d0 con 0x7f52980b2130 2024-03-28T01:06:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f528f400640 1 -- 172.21.15.33:0/3755440784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528800aa90 con 0x7f52980b2130 2024-03-28T01:06:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.650+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529814a0b0 con 0x7f52980b2130 2024-03-28T01:06:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.651+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5298075310 con 0x7f52980b2130 2024-03-28T01:06:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.652+0000 7f528f400640 1 -- 172.21.15.33:0/3755440784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f528800ac30 con 0x7f52980b2130 2024-03-28T01:06:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.653+0000 7f528f400640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f527406bbd0 0x7f527406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.653+0000 7f528f400640 1 -- 172.21.15.33:0/3755440784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f528808a9d0 con 0x7f52980b2130 2024-03-28T01:06:28.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.653+0000 7f5296000640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f527406bbd0 0x7f527406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:28.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.654+0000 7f5296000640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f527406bbd0 0x7f527406e090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f52980b6c90 tx=0x7f5280008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.658+0000 7f528f400640 1 -- 172.21.15.33:0/3755440784 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f52880594b0 con 0x7f52980b2130 2024-03-28T01:06:28.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.884+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f529806b470 con 0x7f52980b2130 2024-03-28T01:06:28.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.887+0000 7f528f400640 1 -- 172.21.15.33:0/3755440784 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f528805de20 con 0x7f52980b2130 2024-03-28T01:06:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:28.890 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:23.710537+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f527406bbd0 msgr2=0x7f527406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f527406bbd0 0x7f527406e090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f52980b6c90 tx=0x7f5280008040 comp rx=0 tx=0).stop 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52980b2130 msgr2=0x7f5298140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52980b2130 0x7f5298140770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f528800c730 tx=0x7f528800cbf0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 shutdown_connections 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f527406bbd0 0x7f527406e090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5298148b20 0x7f529813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529813f500 0x7f52981423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 --2- 172.21.15.33:0/3755440784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52980b2130 0x7f5298140770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 >> 172.21.15.33:0/3755440784 conn(0x7f52980b89c0 msgr2=0x7f529813ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 shutdown_connections 2024-03-28T01:06:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:28.889+0000 7f5297e00640 1 -- 172.21.15.33:0/3755440784 wait complete. 2024-03-28T01:06:28.910 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-28T01:06:28.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.100+0000 7f34b5c00640 1 Processor -- start 2024-03-28T01:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.100+0000 7f34b5c00640 1 -- start start 2024-03-28T01:06:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34b5c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00b8d00 0x7f34b014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34b5c00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b00b55b0 con 0x7f34b00b51d0 2024-03-28T01:06:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34b5c00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b00b9190 con 0x7f34b00b6b80 2024-03-28T01:06:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34b5c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b014a820 con 0x7f34b00b8d00 2024-03-28T01:06:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34afe00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f34b00b51d0 0x7f34b00b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41034/0 (socket says 172.21.15.33:41034) 2024-03-28T01:06:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34afe00640 1 -- 172.21.15.33:0/2225314724 learned_addr learned my addr 172.21.15.33:0/2225314724 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:29.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34a7e00640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00b8d00 0x7f34b014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.102+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757822432 0 0) 0x7f34b00b55b0 con 0x7f34b00b51d0 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3494003520 con 0x7f34b00b51d0 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2344008523 0 0) 0x7f3494003520 con 0x7f34b00b51d0 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00b8d00 msgr2=0x7f34b014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00b8d00 0x7f34b014ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 >> v1:172.21.15.194:6789/0 conn(0x7f34b00b6b80 legacy=0x7f34b014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00b9800 con 0x7f34b00b51d0 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f34a0001670 con 0x7f34b00b51d0 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 >> v1:172.21.15.33:6789/0 conn(0x7f34b00b51d0 legacy=0x7f34b00b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34940039e0 0x7f3494003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3494004800 0x7f3494004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940051c0 0x7f349400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349400bf00 con 0x7f34940039e0 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349400c080 con 0x7f3494004800 2024-03-28T01:06:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349400c200 con 0x7f34940051c0 2024-03-28T01:06:29.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.103+0000 7f34afe00640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34940039e0 0x7f3494003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.104+0000 7f34afe00640 1 -- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940051c0 msgr2=0x7f349400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.104+0000 7f34afe00640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940051c0 0x7f349400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.104+0000 7f34afe00640 1 -- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3494004800 msgr2=0x7f3494004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.104+0000 7f34afe00640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3494004800 0x7f3494004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.104+0000 7f34afe00640 1 -- 172.21.15.33:0/2225314724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494003520 con 0x7f34940039e0 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.105+0000 7f34afe00640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34940039e0 0x7f3494003e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c240 tx=0x7f3494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2bac9573f123b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.105+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0007cf0 con 0x7f34940039e0 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.105+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a0030070 con 0x7f34940039e0 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.105+0000 7f34af400640 1 -- 172.21.15.33:0/2225314724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0005d50 con 0x7f34940039e0 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.106+0000 7f34b5c00640 1 -- 172.21.15.33:0/2225314724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34940039e0 msgr2=0x7f3494003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.106+0000 7f34b5c00640 1 --2- 172.21.15.33:0/2225314724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34940039e0 0x7f3494003e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c240 tx=0x7f3494005830 comp rx=0 tx=0).stop 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.106+0000 7f34b5200640 1 -- 172.21.15.33:0/2225314724 reap_dead start 2024-03-28T01:06:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.106+0000 7f34b5c00640 1 -- 172.21.15.33:0/2225314724 shutdown_connections 2024-03-28T01:06:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.106+0000 7f34b5c00640 1 -- 172.21.15.33:0/2225314724 >> 172.21.15.33:0/2225314724 conn(0x7f34b0072940 msgr2=0x7f34b0072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.106+0000 7f34b5c00640 1 -- 172.21.15.33:0/2225314724 shutdown_connections 2024-03-28T01:06:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.106+0000 7f34b5c00640 1 -- 172.21.15.33:0/2225314724 wait complete. 2024-03-28T01:06:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.107+0000 7f34b5c00640 1 Processor -- start 2024-03-28T01:06:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.108+0000 7f34b5c00640 1 -- start start 2024-03-28T01:06:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.108+0000 7f34b5c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940039e0 0x7f34b01498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.108+0000 7f34b5c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b01dfc60 0x7f34b00b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.108+0000 7f34b5c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b01e00e0 0x7f34b00b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.108+0000 7f34b5c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00b39c0 con 0x7f34b01e00e0 2024-03-28T01:06:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.108+0000 7f34b5c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00b44c0 con 0x7f34b01dfc60 2024-03-28T01:06:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.108+0000 7f34b5c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b014ac70 con 0x7f34940039e0 2024-03-28T01:06:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b01e00e0 0x7f34b00b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b01e00e0 0x7f34b00b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36302/0 (socket says 172.21.15.33:36302) 2024-03-28T01:06:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 -- 172.21.15.33:0/933934055 learned_addr learned my addr 172.21.15.33:0/933934055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34afe00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940039e0 0x7f34b01498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34a7e00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b01dfc60 0x7f34b00b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 -- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940039e0 msgr2=0x7f34b01498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940039e0 0x7f34b01498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 -- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b01dfc60 msgr2=0x7f34b00b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b01dfc60 0x7f34b00b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.109+0000 7f34b5200640 1 -- 172.21.15.33:0/933934055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b01dca20 con 0x7f34b01e00e0 2024-03-28T01:06:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.110+0000 7f34b5200640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b01e00e0 0x7f34b00b20c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34b01472b0 tx=0x7f34a800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.110+0000 7f34acc00640 1 -- 172.21.15.33:0/933934055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8017070 con 0x7f34b01e00e0 2024-03-28T01:06:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.110+0000 7f34acc00640 1 -- 172.21.15.33:0/933934055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a8005d60 con 0x7f34b01e00e0 2024-03-28T01:06:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.110+0000 7f34acc00640 1 -- 172.21.15.33:0/933934055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a800a7f0 con 0x7f34b01e00e0 2024-03-28T01:06:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.110+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b01dcd00 con 0x7f34b01e00e0 2024-03-28T01:06:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.110+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b00aa0e0 con 0x7f34b01e00e0 2024-03-28T01:06:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.112+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b00b8ac0 con 0x7f34b01e00e0 2024-03-28T01:06:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.112+0000 7f34acc00640 1 -- 172.21.15.33:0/933934055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f34a800a990 con 0x7f34b01e00e0 2024-03-28T01:06:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.113+0000 7f34acc00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f348c06bbd0 0x7f348c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.113+0000 7f34acc00640 1 -- 172.21.15.33:0/933934055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34a8089a90 con 0x7f34b01e00e0 2024-03-28T01:06:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.114+0000 7f34afe00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f348c06bbd0 0x7f348c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.114+0000 7f34afe00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f348c06bbd0 0x7f348c06e090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034000 tx=0x7f34a0007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.118+0000 7f34acc00640 1 -- 172.21.15.33:0/933934055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f34b00b8ac0 con 0x7f34b01e00e0 2024-03-28T01:06:29.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.361+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34b0075310 con 0x7f34b01e00e0 2024-03-28T01:06:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.363+0000 7f34acc00640 1 -- 172.21.15.33:0/933934055 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f34a8057fc0 con 0x7f34b01e00e0 2024-03-28T01:06:29.366 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:29.366 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:23.710537+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f348c06bbd0 msgr2=0x7f348c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f348c06bbd0 0x7f348c06e090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034000 tx=0x7f34a0007000 comp rx=0 tx=0).stop 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b01e00e0 msgr2=0x7f34b00b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b01e00e0 0x7f34b00b20c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34b01472b0 tx=0x7f34a800cb90 comp rx=0 tx=0).stop 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 shutdown_connections 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f348c06bbd0 0x7f348c06e090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b01e00e0 0x7f34b00b20c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b01dfc60 0x7f34b00b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 --2- 172.21.15.33:0/933934055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34940039e0 0x7f34b01498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.365+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 >> 172.21.15.33:0/933934055 conn(0x7f34b0072940 msgr2=0x7f34b0073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.366+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 shutdown_connections 2024-03-28T01:06:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.366+0000 7f34b5c00640 1 -- 172.21.15.33:0/933934055 wait complete. 2024-03-28T01:06:29.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-28T01:06:29.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-28T01:06:29.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-28T01:06:29.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-28T01:06:29.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-28T01:06:29.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-28T01:06:29.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-28T01:06:29.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-28T01:06:29.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.705+0000 7fc79ea00640 1 Processor -- start 2024-03-28T01:06:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.707+0000 7fc79ea00640 1 -- start start 2024-03-28T01:06:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.708+0000 7fe7da000640 1 Processor -- start 2024-03-28T01:06:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.708+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc79003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.709+0000 7fe7da000640 1 -- start start 2024-03-28T01:06:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.709+0000 7fc79ea00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7900ccbb0 0x7fc7900c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fc79ea00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7900395a0 con 0x7fc7900cc770 2024-03-28T01:06:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.709+0000 7fc797e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc79003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fc79ea00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79003a900 con 0x7fc7900ccbb0 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fc79ea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79003acc0 con 0x7fc790038870 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fc797e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc79003a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:47980/0 (socket says 172.21.15.33:47980) 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fc797e00640 1 -- 172.21.15.33:0/462635132 learned_addr learned my addr 172.21.15.33:0/462635132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fe7da000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7cc03a3c0 0x7fe7cc0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fe7da000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0cce80 con 0x7fe7cc0c6950 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fe7da000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0395a0 con 0x7fe7cc038870 2024-03-28T01:06:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.710+0000 7fe7da000640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc03acc0 con 0x7fe7cc03a3c0 2024-03-28T01:06:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.709+0000 7fc797400640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7fc7900cc770 0x7fc7900c6410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37854/0 (socket says 172.21.15.33:37854) 2024-03-28T01:06:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.711+0000 7fc79ca00640 1 --2- 172.21.15.33:0/462635132 >> v2:172.21.15.33:3300/0 conn(0x7fc7900ccbb0 0x7fc7900c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.711+0000 7fe7d3400640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7fe7cc038870 0x7fe7cc0cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37858/0 (socket says 172.21.15.33:37858) 2024-03-28T01:06:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.711+0000 7fe7d3400640 1 -- 172.21.15.33:0/2168249703 learned_addr learned my addr 172.21.15.33:0/2168249703 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.712+0000 7fc797e00640 1 -- 172.21.15.33:0/462635132 >> v2:172.21.15.33:3300/0 conn(0x7fc7900ccbb0 msgr2=0x7fc7900c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.712+0000 7fc796a00640 1 -- 172.21.15.33:0/462635132 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684315792 0 0) 0x7fc7900395a0 con 0x7fc7900cc770 2024-03-28T01:06:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.713+0000 7fe7d2a00640 1 --2- 172.21.15.33:0/2168249703 >> v2:172.21.15.33:3300/0 conn(0x7fe7cc03a3c0 0x7fe7cc0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.714+0000 7fe7d2a00640 1 -- 172.21.15.33:0/2168249703 >> v1:172.21.15.194:6789/0 conn(0x7fe7cc038870 legacy=0x7fe7cc0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.712+0000 7fc797e00640 1 --2- 172.21.15.33:0/462635132 >> v2:172.21.15.33:3300/0 conn(0x7fc7900ccbb0 0x7fc7900c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.712+0000 7fc797e00640 1 -- 172.21.15.33:0/462635132 >> v1:172.21.15.194:6789/0 conn(0x7fc7900cc770 legacy=0x7fc7900c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.712+0000 7fc797e00640 1 -- 172.21.15.33:0/462635132 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790038c50 con 0x7fc790038870 2024-03-28T01:06:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.714+0000 7fc79ca00640 1 --2- 172.21.15.33:0/462635132 >> v2:172.21.15.33:3300/0 conn(0x7fc7900ccbb0 0x7fc7900c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.714+0000 7f8593200640 1 Processor -- start 2024-03-28T01:06:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.715+0000 7f8593200640 1 -- start start 2024-03-28T01:06:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.715+0000 7fc797e00640 1 --2- 172.21.15.33:0/462635132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc79003a3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009870 tx=0x7fc78c02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738b8c89cf42fca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.714+0000 7fe7d2a00640 1 -- 172.21.15.33:0/2168249703 >> v1:172.21.15.33:6789/0 conn(0x7fe7cc0c6950 legacy=0x7fe7cc0d7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.714+0000 7fe7d2000640 1 -- 172.21.15.33:0/2168249703 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888670091 0 0) 0x7fe7cc0395a0 con 0x7fe7cc038870 2024-03-28T01:06:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.714+0000 7fe7d2a00640 1 -- 172.21.15.33:0/2168249703 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc038c50 con 0x7fe7cc03a3c0 2024-03-28T01:06:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7fc796a00640 1 -- 172.21.15.33:0/462635132 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c034070 con 0x7fc790038870 2024-03-28T01:06:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7fe7d2a00640 1 --2- 172.21.15.33:0/2168249703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc03a3c0 0x7fe7cc0c6410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc03ae40 tx=0x7fe7c002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206b8013e19da4b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7fc796a00640 1 -- 172.21.15.33:0/462635132 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c030880 con 0x7fc790038870 2024-03-28T01:06:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7fe7d2000640 1 -- 172.21.15.33:0/2168249703 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0037070 con 0x7fe7cc03a3c0 2024-03-28T01:06:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7f8593200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c038380 0x7f858c0d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7f8593200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f858c0d4970 0x7f858c0ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7f8593200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c036f40 con 0x7f858c036b60 2024-03-28T01:06:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7fe7d2000640 1 -- 172.21.15.33:0/2168249703 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c0030710 con 0x7fe7cc03a3c0 2024-03-28T01:06:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7fe7d2000640 1 -- 172.21.15.33:0/2168249703 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0030ae0 con 0x7fe7cc03a3c0 2024-03-28T01:06:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7f8593200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c0d0b70 con 0x7f858c0d4970 2024-03-28T01:06:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.716+0000 7f8593200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0d0e50 con 0x7f858c038380 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7fc796a00640 1 -- 172.21.15.33:0/462635132 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c034070 con 0x7fc790038870 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7fc79ea00640 1 -- 172.21.15.33:0/462635132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 msgr2=0x7fc79003a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7fc79ea00640 1 --2- 172.21.15.33:0/462635132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc79003a3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009870 tx=0x7fc78c02b9c0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fc79ea00640 1 -- 172.21.15.33:0/462635132 shutdown_connections 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fc79ea00640 1 --2- 172.21.15.33:0/462635132 >> v2:172.21.15.33:3300/0 conn(0x7fc7900ccbb0 0x7fc7900c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fc79ea00640 1 --2- 172.21.15.33:0/462635132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc79003a3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fc79ea00640 1 -- 172.21.15.33:0/462635132 >> 172.21.15.33:0/462635132 conn(0x7fc79003ea60 msgr2=0x7fc79003ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fc79ea00640 1 -- 172.21.15.33:0/462635132 shutdown_connections 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7f8591e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f858c036b60 0x7f858c0d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41048/0 (socket says 172.21.15.33:41048) 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.717+0000 7f8591e00640 1 -- 172.21.15.33:0/2663227962 learned_addr learned my addr 172.21.15.33:0/2663227962 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7f8591400640 1 --2- 172.21.15.33:0/2663227962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c038380 0x7f858c0d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fc79ea00640 1 -- 172.21.15.33:0/462635132 wait complete. 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7f8591400640 1 -- 172.21.15.33:0/2663227962 >> v2:172.21.15.33:3300/0 conn(0x7f858c0d4970 msgr2=0x7f858c0ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7f8591400640 1 --2- 172.21.15.33:0/2663227962 >> v2:172.21.15.33:3300/0 conn(0x7f858c0d4970 0x7f858c0ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7f8591400640 1 -- 172.21.15.33:0/2663227962 >> v1:172.21.15.33:6789/0 conn(0x7f858c036b60 legacy=0x7f858c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7f8591400640 1 -- 172.21.15.33:0/2663227962 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c034e00 con 0x7f858c038380 2024-03-28T01:06:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7f11e6c00640 1 Processor -- start 2024-03-28T01:06:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.720+0000 7f11e6c00640 1 -- start start 2024-03-28T01:06:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fe7da000640 1 -- 172.21.15.33:0/2168249703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc03a3c0 msgr2=0x7fe7cc0c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.718+0000 7fe7da000640 1 --2- 172.21.15.33:0/2168249703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc03a3c0 0x7fe7cc0c6410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc03ae40 tx=0x7fe7c002b790 comp rx=0 tx=0).stop 2024-03-28T01:06:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.720+0000 7fe7da000640 1 -- 172.21.15.33:0/2168249703 shutdown_connections 2024-03-28T01:06:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.720+0000 7fe7da000640 1 --2- 172.21.15.33:0/2168249703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc03a3c0 0x7fe7cc0c6410 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc03ae40 tx=0x7fe7c002b790 comp rx=0 tx=0).stop 2024-03-28T01:06:29.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.720+0000 7fe7da000640 1 -- 172.21.15.33:0/2168249703 >> 172.21.15.33:0/2168249703 conn(0x7fe7cc03ea60 msgr2=0x7fe7cc03ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.721+0000 7f8591400640 1 --2- 172.21.15.33:0/2663227962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c038380 0x7f858c0d4430 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85840097a0 tx=0x7f858402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f2534710a98a581 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.721+0000 7fe7da000640 1 -- 172.21.15.33:0/2168249703 shutdown_connections 2024-03-28T01:06:29.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.721+0000 7fe7da000640 1 -- 172.21.15.33:0/2168249703 wait complete. 2024-03-28T01:06:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.722+0000 7fc79ea00640 1 Processor -- start 2024-03-28T01:06:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.722+0000 7fc79ea00640 1 -- start start 2024-03-28T01:06:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.722+0000 7f8590a00640 1 -- 172.21.15.33:0/2663227962 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584034070 con 0x7f858c038380 2024-03-28T01:06:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.722+0000 7f11e6c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0148cf0 0x7f11e0150860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.722+0000 7f11e6c00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e00b2510 con 0x7f11e013f500 2024-03-28T01:06:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.722+0000 7f11e6c00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e00b4670 con 0x7f11e00b2130 2024-03-28T01:06:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.722+0000 7f11e6c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00b1cd0 con 0x7f11e0148cf0 2024-03-28T01:06:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.723+0000 7f11e4e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11e013f500 0x7f11e01485e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41052/0 (socket says 172.21.15.33:41052) 2024-03-28T01:06:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.723+0000 7f11e4e00640 1 -- 172.21.15.33:0/2503769609 learned_addr learned my addr 172.21.15.33:0/2503769609 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.723+0000 7f8590a00640 1 -- 172.21.15.33:0/2663227962 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8584030610 con 0x7f858c038380 2024-03-28T01:06:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.724+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc7900cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.724+0000 7f8590a00640 1 -- 172.21.15.33:0/2663227962 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584034070 con 0x7f858c038380 2024-03-28T01:06:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.724+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7900cc770 0x7fc7900c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.724+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ccbb0 0x7fc790161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.724+0000 7f11e6200640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0148cf0 0x7f11e0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.724+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448044279 0 0) 0x7f11e00b2510 con 0x7f11e013f500 2024-03-28T01:06:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.724+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c4003520 con 0x7f11e013f500 2024-03-28T01:06:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7fc797e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc7900cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7fc797e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc7900cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:47998/0 (socket says 172.21.15.33:47998) 2024-03-28T01:06:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7f8593200640 1 -- 172.21.15.33:0/2663227962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c038380 msgr2=0x7f858c0d4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7f8593200640 1 --2- 172.21.15.33:0/2663227962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c038380 0x7f858c0d4430 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85840097a0 tx=0x7f858402b960 comp rx=0 tx=0).stop 2024-03-28T01:06:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856736200 0 0) 0x7f11c4003520 con 0x7f11e013f500 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7fc797400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7900cc770 0x7fc7900c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7fc797400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7900cc770 0x7fc7900c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44282/0 (socket says 172.21.15.33:44282) 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.726+0000 7fc79ca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ccbb0 0x7fc790161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.726+0000 7fc79ca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ccbb0 0x7fc790161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36338/0 (socket says 172.21.15.33:36338) 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.726+0000 7fc797e00640 1 -- 172.21.15.33:0/195730853 learned_addr learned my addr 172.21.15.33:0/195730853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.726+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79003b150 con 0x7fc7900ccbb0 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc790039840 con 0x7fc7900cc770 2024-03-28T01:06:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900cf970 con 0x7fc790038870 2024-03-28T01:06:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7fc2b5600640 1 Processor -- start 2024-03-28T01:06:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7fc2b5600640 1 -- start start 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.725+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0148cf0 msgr2=0x7f11e0150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0148cf0 0x7f11e0150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 >> v1:172.21.15.194:6789/0 conn(0x7f11e00b2130 legacy=0x7f11e013edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e006bef0 con 0x7f11e013f500 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f11d80019f0 con 0x7f11e013f500 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 >> v1:172.21.15.33:6789/0 conn(0x7f11e013f500 legacy=0x7f11e01485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c40039e0 0x7f11c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11c4004800 0x7f11c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11c4005290 0x7f11c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00b1cd0 con 0x7f11c40039e0 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c400c1b0 con 0x7f11c4005290 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c400c3b0 con 0x7f11c4004800 2024-03-28T01:06:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.729+0000 7f11e4e00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11c4004800 0x7f11c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7fc79ca00640 1 -- 172.21.15.33:0/195730853 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 msgr2=0x7fc7900cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7fc79ca00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc7900cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7fc79ca00640 1 -- 172.21.15.33:0/195730853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7900cc770 msgr2=0x7fc7900c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7fc79ca00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7900cc770 0x7fc7900c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7fc79ca00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900c5b70 con 0x7fc7900ccbb0 2024-03-28T01:06:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.729+0000 7fc797400640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7900cc770 0x7fc7900c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.729+0000 7fc79ca00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ccbb0 0x7fc790161c70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc79806ab70 tx=0x7fc798068b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.729+0000 7fc2b5600640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2a803ae20 0x7fc2a80c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc797e00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc790038870 0x7fc7900cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc2b5600640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a803b280 con 0x7fc2a80c6740 2024-03-28T01:06:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc2b5600640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a80cc730 con 0x7fc2a803a370 2024-03-28T01:06:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc2b5600640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a80cca60 con 0x7fc2a803ae20 2024-03-28T01:06:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc2af400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2a803ae20 0x7fc2a80c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fe7da000640 1 Processor -- start 2024-03-28T01:06:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7f8593200640 1 -- 172.21.15.33:0/2663227962 shutdown_connections 2024-03-28T01:06:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7f8593200640 1 --2- 172.21.15.33:0/2663227962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c038380 0x7f858c0d4430 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7f8593200640 1 --2- 172.21.15.33:0/2663227962 >> v2:172.21.15.33:3300/0 conn(0x7f858c0d4970 0x7f858c0ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7f8593200640 1 -- 172.21.15.33:0/2663227962 >> 172.21.15.33:0/2663227962 conn(0x7f858c03a8a0 msgr2=0x7f858c03acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.727+0000 7f8593200640 1 -- 172.21.15.33:0/2663227962 shutdown_connections 2024-03-28T01:06:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.728+0000 7f8593200640 1 -- 172.21.15.33:0/2663227962 wait complete. 2024-03-28T01:06:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7f11e6200640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11c4005290 0x7f11c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc2af400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2a803ae20 0x7fc2a80c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36348/0 (socket says 172.21.15.33:36348) 2024-03-28T01:06:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc2af400640 1 -- 172.21.15.33:0/2699726405 learned_addr learned my addr 172.21.15.33:0/2699726405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798091070 con 0x7fc7900ccbb0 2024-03-28T01:06:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7f8593200640 1 Processor -- start 2024-03-28T01:06:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc756400640 1 Processor -- start 2024-03-28T01:06:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7f8593200640 1 -- start start 2024-03-28T01:06:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.730+0000 7fe7da000640 1 -- start start 2024-03-28T01:06:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc2af400640 1 -- 172.21.15.33:0/2699726405 >> v1:172.21.15.194:6789/0 conn(0x7fc2a803a370 legacy=0x7fc2a80ccef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T01:06:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc2af400640 1 -- 172.21.15.33:0/2699726405 >> v1:172.21.15.33:6789/0 conn(0x7fc2a80c6740 legacy=0x7fc2a80d6ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T01:06:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc2af400640 1 -- 172.21.15.33:0/2699726405 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a803a750 con 0x7fc2a803ae20 2024-03-28T01:06:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc756400640 1 -- start start 2024-03-28T01:06:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7900cf690 con 0x7fc7900ccbb0 2024-03-28T01:06:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc798073b30 con 0x7fc7900ccbb0 2024-03-28T01:06:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7900375e0 con 0x7fc7900ccbb0 2024-03-28T01:06:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798096660 con 0x7fc7900ccbb0 2024-03-28T01:06:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.732+0000 7f8593200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c036b60 0x7f858c0d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.732+0000 7f8593200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c038380 0x7f858c0d6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.732+0000 7f8593200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f858c0d4970 0x7f858c0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7f11e5800640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c40039e0 0x7f11c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7f11e4e00640 1 -- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11c4005290 msgr2=0x7f11c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7f11e4e00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11c4005290 0x7f11c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7f11e4e00640 1 -- 172.21.15.33:0/2503769609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c40039e0 msgr2=0x7f11c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7f11e4e00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c40039e0 0x7f11c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.731+0000 7f11e4e00640 1 -- 172.21.15.33:0/2503769609 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e006bef0 con 0x7f11c4004800 2024-03-28T01:06:29.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.732+0000 7fc756400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc74803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7fc756400640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7480395a0 con 0x7fc7480cc770 2024-03-28T01:06:29.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7fc756400640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc748038c50 con 0x7fc7480ccbb0 2024-03-28T01:06:29.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7fc756400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74803a900 con 0x7fc748038870 2024-03-28T01:06:29.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7f8591e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c036b60 0x7f858c0d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7f8591e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c036b60 0x7f858c0d7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48008/0 (socket says 172.21.15.33:48008) 2024-03-28T01:06:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7f8593200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0cafa0 con 0x7f858c038380 2024-03-28T01:06:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7f8593200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0cae20 con 0x7f858c0d4970 2024-03-28T01:06:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7f8593200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0db2e0 con 0x7f858c036b60 2024-03-28T01:06:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7f8591e00640 1 -- 172.21.15.33:0/1706435754 learned_addr learned my addr 172.21.15.33:0/1706435754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.734+0000 7f8592800640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f858c0d4970 0x7f858c0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7f11e5800640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c40039e0 0x7f11c4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.733+0000 7fc2af400640 1 --2- 172.21.15.33:0/2699726405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803ae20 0x7fc2a80c6200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009870 tx=0x7fc29c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591d5e7bc9d92c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.734+0000 7f8591400640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c038380 0x7f858c0d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.734+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc774001030 con 0x7fc7900ccbb0 2024-03-28T01:06:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.734+0000 7fe7da000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc038870 0x7fe7cc0bf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fc755000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc74803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fe7da000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7cc0c6950 0x7fe7cc1624e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fe7da000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7cc0bf550 0x7fe7cc162a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fe7da000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc03a800 con 0x7fe7cc038870 2024-03-28T01:06:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fe7da000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc039880 con 0x7fe7cc0c6950 2024-03-28T01:06:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fe7da000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc03b150 con 0x7fe7cc0bf550 2024-03-28T01:06:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7f8591e00640 1 -- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f858c0d4970 msgr2=0x7f858c0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fc2aea00640 1 -- 172.21.15.33:0/2699726405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c038070 con 0x7fc2a803ae20 2024-03-28T01:06:29.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7f11e4e00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11c4004800 0x7f11c4004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11d800a8c0 tx=0x7f11c4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e3a5d92fac988e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fc2aea00640 1 -- 172.21.15.33:0/2699726405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c02f050 con 0x7fc2a803ae20 2024-03-28T01:06:29.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fc2aea00640 1 -- 172.21.15.33:0/2699726405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c031970 con 0x7fc2a803ae20 2024-03-28T01:06:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7f8591e00640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f858c0d4970 0x7f858c0c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7f8591e00640 1 -- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c038380 msgr2=0x7f858c0d6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7f8591e00640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c038380 0x7f858c0d6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7f8591e00640 1 -- 172.21.15.33:0/1706435754 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0d9400 con 0x7f858c036b60 2024-03-28T01:06:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7f8591400640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c038380 0x7f858c0d6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800a4c0 con 0x7f11c4004800 2024-03-28T01:06:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d8034650 con 0x7f11c4004800 2024-03-28T01:06:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7f11dfe00640 1 -- 172.21.15.33:0/2503769609 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8034a40 con 0x7f11c4004800 2024-03-28T01:06:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fc755000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc74803a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48016/0 (socket says 172.21.15.33:48016) 2024-03-28T01:06:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.735+0000 7fc755000640 1 -- 172.21.15.33:0/1920736879 learned_addr learned my addr 172.21.15.33:0/1920736879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7fe7d3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc038870 0x7fe7cc0bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7fe7d3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc038870 0x7fe7cc0bf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36366/0 (socket says 172.21.15.33:36366) 2024-03-28T01:06:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7fe7d3400640 1 -- 172.21.15.33:0/1886665066 learned_addr learned my addr 172.21.15.33:0/1886665066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7f8591e00640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c036b60 0x7f858c0d7f80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f858000c480 tx=0x7f858000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7fc755000640 1 -- 172.21.15.33:0/1920736879 >> v1:172.21.15.194:6789/0 conn(0x7fc7480ccbb0 legacy=0x7fc7480d7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7fc755000640 1 -- 172.21.15.33:0/1920736879 >> v1:172.21.15.33:6789/0 conn(0x7fc7480cc770 legacy=0x7fc7480c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7fc755000640 1 -- 172.21.15.33:0/1920736879 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74803c110 con 0x7fc748038870 2024-03-28T01:06:29.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.736+0000 7fe7d3e00640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7cc0bf550 0x7fe7cc162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7f11e6c00640 1 -- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11c4004800 msgr2=0x7f11c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7f11e6c00640 1 --2- 172.21.15.33:0/2503769609 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11c4004800 0x7f11c4004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11d800a8c0 tx=0x7f11c4005830 comp rx=0 tx=0).stop 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fe7d2a00640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7cc0c6950 0x7fe7cc1624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fe7d3400640 1 -- 172.21.15.33:0/1886665066 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7cc0bf550 msgr2=0x7fe7cc162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fe7d3400640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7cc0bf550 0x7fe7cc162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fe7d3400640 1 -- 172.21.15.33:0/1886665066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7cc0c6950 msgr2=0x7fe7cc1624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fe7d3400640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7cc0c6950 0x7fe7cc1624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fe7d3400640 1 -- 172.21.15.33:0/1886665066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc162030 con 0x7fe7cc038870 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7f11e6200640 1 -- 172.21.15.33:0/2503769609 reap_dead start 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7f11e6c00640 1 -- 172.21.15.33:0/2503769609 shutdown_connections 2024-03-28T01:06:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7f11e6c00640 1 -- 172.21.15.33:0/2503769609 >> 172.21.15.33:0/2503769609 conn(0x7f11e00b89c0 msgr2=0x7f11e00b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fe7d3400640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc038870 0x7fe7cc0bf010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c570 tx=0x7fe7c800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580018070 con 0x7f858c036b60 2024-03-28T01:06:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8580005ce0 con 0x7f858c036b60 2024-03-28T01:06:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85800072d0 con 0x7f858c036b60 2024-03-28T01:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7f11e6c00640 1 -- 172.21.15.33:0/2503769609 shutdown_connections 2024-03-28T01:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8018070 con 0x7fe7cc038870 2024-03-28T01:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7f11e6c00640 1 -- 172.21.15.33:0/2503769609 wait complete. 2024-03-28T01:06:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c80074e0 con 0x7fe7cc038870 2024-03-28T01:06:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80049e0 con 0x7fe7cc038870 2024-03-28T01:06:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc798096800 con 0x7fc7900ccbb0 2024-03-28T01:06:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc0c04d0 con 0x7fe7cc038870 2024-03-28T01:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc0be950 con 0x7fe7cc038870 2024-03-28T01:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.739+0000 7fc755000640 1 --2- 172.21.15.33:0/1920736879 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc74803a3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098a0 tx=0x7fc73c02ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4455c5ab24f2f11f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.738+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c0d9670 con 0x7f858c036b60 2024-03-28T01:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.739+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c16ef30 con 0x7f858c036b60 2024-03-28T01:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.739+0000 7fc74f400640 1 -- 172.21.15.33:0/1920736879 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c02f850 con 0x7fc748038870 2024-03-28T01:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.739+0000 7fc74f400640 1 -- 172.21.15.33:0/1920736879 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c02f9f0 con 0x7fc748038870 2024-03-28T01:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.739+0000 7fc74f400640 1 -- 172.21.15.33:0/1920736879 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c02fe00 con 0x7fc748038870 2024-03-28T01:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.740+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7cc0057d0 con 0x7fe7cc038870 2024-03-28T01:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fc2b5600640 1 -- 172.21.15.33:0/2699726405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803ae20 msgr2=0x7fc2a80c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.737+0000 7fc2b5600640 1 --2- 172.21.15.33:0/2699726405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803ae20 0x7fc2a80c6200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009870 tx=0x7fc29c02b7f0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.741+0000 7fc2b5600640 1 -- 172.21.15.33:0/2699726405 shutdown_connections 2024-03-28T01:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.741+0000 7fc2b5600640 1 --2- 172.21.15.33:0/2699726405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803ae20 0x7fc2a80c6200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.741+0000 7fc2b5600640 1 -- 172.21.15.33:0/2699726405 >> 172.21.15.33:0/2699726405 conn(0x7fc2a803ec80 msgr2=0x7fc2a803f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.741+0000 7fc78be00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc77806bc00 0x7fc77806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.741+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f858c005390 con 0x7f858c036b60 2024-03-28T01:06:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.741+0000 7fc756400640 1 -- 172.21.15.33:0/1920736879 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 msgr2=0x7fc74803a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.741+0000 7fc756400640 1 --2- 172.21.15.33:0/1920736879 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc74803a3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098a0 tx=0x7fc73c02ba30 comp rx=0 tx=0).stop 2024-03-28T01:06:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7fc756400640 1 -- 172.21.15.33:0/1920736879 shutdown_connections 2024-03-28T01:06:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7fc756400640 1 --2- 172.21.15.33:0/1920736879 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc74803a3c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7fc756400640 1 -- 172.21.15.33:0/1920736879 >> 172.21.15.33:0/1920736879 conn(0x7fc74803ea60 msgr2=0x7fc74803ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7fc2b5600640 1 -- 172.21.15.33:0/2699726405 shutdown_connections 2024-03-28T01:06:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7f11e6c00640 1 Processor -- start 2024-03-28T01:06:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc798105800 con 0x7fc7900ccbb0 2024-03-28T01:06:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7fc797e00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc77806bc00 0x7fc77806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.742+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f858001b070 con 0x7f858c036b60 2024-03-28T01:06:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.743+0000 7fc756400640 1 -- 172.21.15.33:0/1920736879 shutdown_connections 2024-03-28T01:06:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.743+0000 7fc2b5600640 1 -- 172.21.15.33:0/2699726405 wait complete. 2024-03-28T01:06:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.743+0000 7fc756400640 1 -- 172.21.15.33:0/1920736879 wait complete. 2024-03-28T01:06:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.744+0000 7f858a000640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f857406bc20 0x7f857406e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.743+0000 7fc78be00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7fc778073e20 0x7fc778076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.744+0000 7f11e6c00640 1 -- start start 2024-03-28T01:06:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.745+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7780768f0 con 0x7fc778073e20 2024-03-28T01:06:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.744+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8580089d20 con 0x7f858c036b60 2024-03-28T01:06:29.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.745+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc798105bf0 con 0x7fc7900ccbb0 2024-03-28T01:06:29.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.744+0000 7f858a000640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7f8574073ea0 0x7f85740762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.745+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 --> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8580013e50 con 0x7f8574073ea0 2024-03-28T01:06:29.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.745+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858008f070 con 0x7f858c036b60 2024-03-28T01:06:29.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.745+0000 7fc756400640 1 Processor -- start 2024-03-28T01:06:29.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7fc797400640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7fc778073e20 0x7fc778076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.745+0000 7fc2b5600640 1 Processor -- start 2024-03-28T01:06:29.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7fc2b5600640 1 -- start start 2024-03-28T01:06:29.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7f8a8de00640 1 Processor -- start 2024-03-28T01:06:29.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7f8591400640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f857406bc20 0x7f857406e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7f8592800640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7f8574073ea0 0x7f85740762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7f11e6c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0137860 0x7f11e01e4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7f11e6c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0133fa0 0x7f11e0134420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7fc797e00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc77806bc00 0x7fc77806e0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009630 tx=0x7fc78c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7fc797400640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7fc778073e20 0x7fc778076240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e5800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0137860 0x7f11e01e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e6c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11e0135db0 0x7f11e0136230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe7c8004b80 con 0x7fe7cc038870 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f8592800640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7f8574073ea0 0x7f85740762c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e5800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0137860 0x7f11e01e4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48040/0 (socket says 172.21.15.33:48040) 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e6c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00b4ec0 con 0x7f11e0133fa0 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e6c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e006b6d0 con 0x7f11e0135db0 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e6c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0072d20 con 0x7f11e0137860 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e5800640 1 -- 172.21.15.33:0/3957892877 learned_addr learned my addr 172.21.15.33:0/3957892877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7f11e4e00640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0133fa0 0x7f11e0134420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.746+0000 7fc756400640 1 -- start start 2024-03-28T01:06:29.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7fc756400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc7480c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7fc756400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7480cc770 0x7fc748166c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.747+0000 7fc756400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7480ccbb0 0x7fc748036af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc756400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74803b150 con 0x7fc7480ccbb0 2024-03-28T01:06:29.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== osd.0 v2:172.21.15.33:6817/1178911672 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc7780768f0 con 0x7fc778073e20 2024-03-28T01:06:29.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7f8591400640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f857406bc20 0x7f857406e0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8584036000 tx=0x7f8584008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc755000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc7480c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc2b5600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803a370 0x7fc2a80c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc755000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc7480c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48046/0 (socket says 172.21.15.33:48046) 2024-03-28T01:06:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc755000640 1 -- 172.21.15.33:0/776875024 learned_addr learned my addr 172.21.15.33:0/776875024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc755a00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7480ccbb0 0x7fc748036af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7fc2b5600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a803ae20 0x7fc2a8162050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7f11e6200640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11e0135db0 0x7f11e0136230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74803bce0 con 0x7fc7480cc770 2024-03-28T01:06:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc748166330 con 0x7fc748038870 2024-03-28T01:06:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7f11e5800640 1 -- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11e0135db0 msgr2=0x7f11e0136230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7fc2afe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803a370 0x7fc2a80c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc2af400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a803ae20 0x7fc2a8162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7f11e5800640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11e0135db0 0x7f11e0136230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7f11e5800640 1 -- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0133fa0 msgr2=0x7f11e0134420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7f11e5800640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0133fa0 0x7f11e0134420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7f11e5800640 1 -- 172.21.15.33:0/3957892877 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0070ef0 con 0x7f11e0137860 2024-03-28T01:06:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f11e4e00640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0133fa0 0x7f11e0134420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc755a00640 1 -- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 msgr2=0x7fc7480c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc755a00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc748038870 0x7fc7480c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc755a00640 1 -- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7480cc770 msgr2=0x7fc748166c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T01:06:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc755a00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7480cc770 0x7fc748166c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc755a00640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc748156cd0 con 0x7fc7480ccbb0 2024-03-28T01:06:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.749+0000 7fc2b5600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a80c6740 0x7fc2a80c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc2af400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a803ae20 0x7fc2a8162050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48054/0 (socket says 172.21.15.33:48054) 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.748+0000 7f8a8de00640 1 -- start start 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f8a8de00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f8a880b4780 0x7f8a880b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f8a8de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88072650 0x7f8a8806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f8a8de00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a88148880 0x7f8a88148d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f8a8de00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a880b1070 con 0x7f8a88148880 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f8a8de00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a880b1f10 con 0x7f8a880b4780 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f8a8de00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880b0af0 con 0x7f8a88072650 2024-03-28T01:06:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fe7c7400640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe7b406bbd0 0x7fe7b406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc2af400640 1 -- 172.21.15.33:0/3815660160 learned_addr learned my addr 172.21.15.33:0/3815660160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fc2b4c00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a80c6740 0x7fc2a80c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a803bed0 con 0x7fc2a803a370 2024-03-28T01:06:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80c6bb0 con 0x7fc2a80c6740 2024-03-28T01:06:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8039210 con 0x7fc2a803ae20 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88072650 0x7f8a8806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88072650 0x7f8a8806c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48064/0 (socket says 172.21.15.33:48064) 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f8a87e00640 1 -- 172.21.15.33:0/2818773612 learned_addr learned my addr 172.21.15.33:0/2818773612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f8a8d400640 1 --2- 172.21.15.33:0/2818773612 >> v2:172.21.15.33:3300/0 conn(0x7f8a88148880 0x7f8a88148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f8a8ca00640 1 --2- 172.21.15.33:0/2818773612 >> v2:172.21.15.194:3300/0 conn(0x7f8a880b4780 0x7f8a880b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7c808a060 con 0x7fe7cc038870 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fe7d2a00640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe7b406bbd0 0x7fe7b406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fe7c7400640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7fe7b4073e50 0x7fe7b4076270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7b4076920 con 0x7fe7b4073e50 2024-03-28T01:06:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.750+0000 7fc755a00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7480ccbb0 0x7fc748036af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc750069ea0 tx=0x7fc750068b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7500718a0 con 0x7fc7480ccbb0 2024-03-28T01:06:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f8a87e00640 1 -- 172.21.15.33:0/2818773612 >> v2:172.21.15.194:3300/0 conn(0x7f8a880b4780 msgr2=0x7f8a880b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f8a87e00640 1 --2- 172.21.15.33:0/2818773612 >> v2:172.21.15.194:3300/0 conn(0x7f8a880b4780 0x7f8a880b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7f8a87e00640 1 -- 172.21.15.33:0/2818773612 >> v2:172.21.15.33:3300/0 conn(0x7f8a88148880 msgr2=0x7f8a88148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7c808a450 con 0x7fe7cc038870 2024-03-28T01:06:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== osd.4 v2:172.21.15.194:6802/548012972 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8580013e50 con 0x7f8574073ea0 2024-03-28T01:06:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc748037530 con 0x7fc7480ccbb0 2024-03-28T01:06:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7481699d0 con 0x7fc7480ccbb0 2024-03-28T01:06:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750071a40 con 0x7fc7480ccbb0 2024-03-28T01:06:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7500718a0 con 0x7fc7480ccbb0 2024-03-28T01:06:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7fe7d3e00640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7fe7b4073e50 0x7fe7b4076270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7fc2afe00640 1 -- 172.21.15.33:0/3815660160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a803ae20 msgr2=0x7fc2a8162050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7fe7d3e00640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7fe7b4073e50 0x7fe7b4076270 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.753+0000 7fc2afe00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a803ae20 0x7fc2a8162050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.753+0000 7fc2afe00640 1 -- 172.21.15.33:0/3815660160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a80c6740 msgr2=0x7fc2a80c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7f8a87e00640 1 --2- 172.21.15.33:0/2818773612 >> v2:172.21.15.33:3300/0 conn(0x7f8a88148880 0x7f8a88148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.752+0000 7f8a87e00640 1 -- 172.21.15.33:0/2818773612 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a880b4b60 con 0x7f8a88072650 2024-03-28T01:06:29.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.753+0000 7f8a87e00640 1 --2- 172.21.15.33:0/2818773612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88072650 0x7f8a8806c7d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a8806d960 tx=0x7f8a7002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95eee7e917bda9b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.754+0000 7f8a8d400640 1 --2- 172.21.15.33:0/2818773612 >> v2:172.21.15.33:3300/0 conn(0x7f8a88148880 0x7f8a88148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.754+0000 7fc2afe00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a80c6740 0x7fc2a80c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.754+0000 7fc2afe00640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8162590 con 0x7fc2a803a370 2024-03-28T01:06:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.754+0000 7fc2af400640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a803ae20 0x7fc2a8162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.754+0000 7fc2b4c00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a80c6740 0x7fc2a80c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.755+0000 7fc2afe00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803a370 0x7fc2a80c8850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b410 tx=0x7fc2a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.755+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4015020 con 0x7fc2a803a370 2024-03-28T01:06:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.755+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a4011830 con 0x7fc2a803a370 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.755+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4011c20 con 0x7fc2a803a370 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.755+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a8162820 con 0x7fc2a803a370 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.755+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a80c9bf0 con 0x7fc2a803a370 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.751+0000 7f11e5800640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0137860 0x7f11e01e4350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d7a0 tx=0x7f11d000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.753+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0016070 con 0x7f11e0137860 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.753+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e0137c40 con 0x7f11e0137860 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.753+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d000b8b0 con 0x7f11e0137860 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.754+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0016070 con 0x7f11e0137860 2024-03-28T01:06:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.754+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e0147200 con 0x7f11e0137860 2024-03-28T01:06:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.756+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc72c001030 con 0x7fc7480ccbb0 2024-03-28T01:06:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.756+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc750071c30 con 0x7fc7480ccbb0 2024-03-28T01:06:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.756+0000 7fe7d2a00640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe7b406bbd0 0x7fe7b406e090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0034000 tx=0x7fe7c002f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.756+0000 7f8a87400640 1 -- 172.21.15.33:0/2818773612 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70035020 con 0x7f8a88072650 2024-03-28T01:06:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.756+0000 7f8a87400640 1 -- 172.21.15.33:0/2818773612 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a70031800 con 0x7f8a88072650 2024-03-28T01:06:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.756+0000 7f8a87400640 1 -- 172.21.15.33:0/2818773612 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70031bb0 con 0x7f8a88072650 2024-03-28T01:06:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.757+0000 7f8a8de00640 1 -- 172.21.15.33:0/2818773612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88072650 msgr2=0x7f8a8806c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.757+0000 7f8a8de00640 1 --2- 172.21.15.33:0/2818773612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88072650 0x7f8a8806c7d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a8806d960 tx=0x7f8a7002b8a0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.757+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== osd.3 v2:172.21.15.33:6800/2717396394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe7b4076920 con 0x7fe7b4073e50 2024-03-28T01:06:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.757+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc2a80c3dc0 con 0x7fc2a803a370 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.758+0000 7fc74cc00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc73006c000 0x7fc73006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.758+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7500716a0 con 0x7fc7480ccbb0 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.758+0000 7fc74cc00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7fc730074200 0x7fc730076620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.759+0000 7fc755000640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc73006c000 0x7fc73006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.759+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc730076cd0 con 0x7fc730074200 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.760+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7501092c0 con 0x7fc7480ccbb0 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.760+0000 7fc755000640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc73006c000 0x7fc73006e4c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0090b0 tx=0x7fc73c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.760+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc2a4018020 con 0x7fc2a803a370 2024-03-28T01:06:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.761+0000 7f8a8de00640 1 -- 172.21.15.33:0/2818773612 shutdown_connections 2024-03-28T01:06:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.761+0000 7f8a8de00640 1 --2- 172.21.15.33:0/2818773612 >> v2:172.21.15.33:3300/0 conn(0x7f8a88148880 0x7f8a88148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.761+0000 7f8a8de00640 1 --2- 172.21.15.33:0/2818773612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88072650 0x7f8a8806c7d0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a8806d960 tx=0x7f8a7002b8a0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.761+0000 7f8a8de00640 1 --2- 172.21.15.33:0/2818773612 >> v2:172.21.15.194:3300/0 conn(0x7f8a880b4780 0x7f8a880b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.761+0000 7f8a8de00640 1 -- 172.21.15.33:0/2818773612 >> 172.21.15.33:0/2818773612 conn(0x7f8a880b57b0 msgr2=0x7f8a880b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.761+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f11d0019070 con 0x7f11e0137860 2024-03-28T01:06:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.761+0000 7fc74fe00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7fc730074200 0x7fc730076620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.762+0000 7fc2a3e00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc29006bc20 0x7fc29006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.762+0000 7fc74fe00640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7fc730074200 0x7fc730076620 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.763+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f11e0075310 con 0x7f11e0137860 2024-03-28T01:06:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.763+0000 7fc2af400640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc29006bc20 0x7fc29006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.763+0000 7f11dd600640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f11c006bdc0 0x7f11c006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.764+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11d008a550 con 0x7f11e0137860 2024-03-28T01:06:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.763+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2a40573b0 con 0x7fc2a803a370 2024-03-28T01:06:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.764+0000 7f11dd600640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f11c0073fc0 0x7f11c00763e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.764+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 --> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11d001ee70 con 0x7f11c0073fc0 2024-03-28T01:06:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.764+0000 7f11e4e00640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f11c006bdc0 0x7f11c006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.765+0000 7f8a8de00640 1 -- 172.21.15.33:0/2818773612 shutdown_connections 2024-03-28T01:06:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.765+0000 7fc2a3e00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7fc290073ea0 0x7fc2900762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.765+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2a4006740 con 0x7fc290073ea0 2024-03-28T01:06:29.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.765+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2a40555b0 con 0x7fc2a803a370 2024-03-28T01:06:29.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.766+0000 7f11e4e00640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f11c006bdc0 0x7f11c006e280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f11d8034000 tx=0x7f11d802f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.766+0000 7fc2af400640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc29006bc20 0x7fc29006e0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc29c02b830 tx=0x7fc29c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.766+0000 7fc2b4c00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7fc290073ea0 0x7fc2900762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.766+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== osd.7 v2:172.21.15.194:6824/2849955959 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc730076cd0 con 0x7fc730074200 2024-03-28T01:06:29.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.767+0000 7f11e6200640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f11c0073fc0 0x7f11c00763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.765+0000 7f8a8de00640 1 -- 172.21.15.33:0/2818773612 wait complete. 2024-03-28T01:06:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.767+0000 7f8a8de00640 1 Processor -- start 2024-03-28T01:06:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.767+0000 7fc2b4c00640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7fc290073ea0 0x7fc2900762c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.768+0000 7f11e6200640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f11c0073fc0 0x7f11c00763e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.770+0000 7f8a8de00640 1 -- start start 2024-03-28T01:06:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.770+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== osd.1 v2:172.21.15.33:6816/4011280304 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc2a4006740 con 0x7fc290073ea0 2024-03-28T01:06:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.771+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== osd.5 v2:172.21.15.194:6816/3192443429 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f11d001ee70 con 0x7f11c0073fc0 2024-03-28T01:06:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.774+0000 7f8a8de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a880b4780 0x7f8a88133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.774+0000 7f8a8de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a88148880 0x7f8a88133220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.774+0000 7f8a8de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88137430 0x7f8a88132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.773+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d00587f0 con 0x7f11e0137860 2024-03-28T01:06:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.775+0000 7f8a8de00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880afdc0 con 0x7f8a880b4780 2024-03-28T01:06:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.777+0000 7f8a8ca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a880b4780 0x7f8a88133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.776+0000 7f8a8d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88137430 0x7f8a88132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.777+0000 7f8a8de00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880b0d90 con 0x7f8a88148880 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.775+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a88148880 0x7f8a88133220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.777+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a88148880 0x7f8a88133220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44372/0 (socket says 172.21.15.33:44372) 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.777+0000 7f8a87e00640 1 -- 172.21.15.33:0/50039831 learned_addr learned my addr 172.21.15.33:0/50039831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.777+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880aff40 con 0x7f8a88137430 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.781+0000 7f8a8ca00640 1 -- 172.21.15.33:0/50039831 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88137430 msgr2=0x7f8a88132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.781+0000 7f8a8ca00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88137430 0x7f8a88132100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.781+0000 7f8a8ca00640 1 -- 172.21.15.33:0/50039831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a88148880 msgr2=0x7f8a88133220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.781+0000 7f8a8ca00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a88148880 0x7f8a88133220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.783+0000 7f8a8ca00640 1 -- 172.21.15.33:0/50039831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88132640 con 0x7f8a880b4780 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.783+0000 7f8a8d400640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a88137430 0x7f8a88132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.783+0000 7f8a87e00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a88148880 0x7f8a88133220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.784+0000 7f8a8ca00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a880b4780 0x7f8a88133c50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000e870 tx=0x7f8a8000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.785+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80017070 con 0x7f8a880b4780 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.785+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000ee60 con 0x7f8a880b4780 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.785+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a881342b0 con 0x7f8a880b4780 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.785+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000cae0 con 0x7f8a880b4780 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.786+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a88131900 con 0x7f8a880b4780 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.785+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 --> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f858c0d1410 con 0x7f8574073ea0 2024-03-28T01:06:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.787+0000 7f858a000640 1 -- 172.21.15.33:0/1706435754 <== osd.4 v2:172.21.15.194:6802/548012972 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f858c0d1410 con 0x7f8574073ea0 2024-03-28T01:06:29.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.788+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a60001030 con 0x7f8a880b4780 2024-03-28T01:06:29.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.790+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8a8001a070 con 0x7f8a880b4780 2024-03-28T01:06:29.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.792+0000 7f8a84c00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8a6406bca0 0x7f8a6406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.792+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc774002dd0 con 0x7fc778073e20 2024-03-28T01:06:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.792+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a8008a720 con 0x7f8a880b4780 2024-03-28T01:06:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.792+0000 7f8a84c00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7f8a64073f20 0x7f8a64076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.792+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a640769f0 con 0x7f8a64073f20 2024-03-28T01:06:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.792+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a80091050 con 0x7f8a880b4780 2024-03-28T01:06:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.793+0000 7f8a87e00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8a6406bca0 0x7f8a6406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.793+0000 7fc78be00640 1 -- 172.21.15.33:0/195730853 <== osd.0 v2:172.21.15.33:6817/1178911672 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc774002dd0 con 0x7fc778073e20 2024-03-28T01:06:29.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7f8574073ea0 msgr2=0x7f85740762c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7f8574073ea0 0x7f85740762c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7fc778073e20 msgr2=0x7fc778076240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7fc79ea00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7fc778073e20 0x7fc778076240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8a87e00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8a6406bca0 0x7f8a6406e160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a7002b8e0 tx=0x7f8a70008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f857406bc20 msgr2=0x7f857406e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f857406bc20 0x7f857406e0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8584036000 tx=0x7f8584008030 comp rx=0 tx=0).stop 2024-03-28T01:06:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c036b60 msgr2=0x7f858c0d7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 --2- 172.21.15.33:0/1706435754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f858c036b60 0x7f858c0d7f80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f858000c480 tx=0x7f858000c940 comp rx=0 tx=0).stop 2024-03-28T01:06:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc77806bc00 msgr2=0x7fc77806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7fc79ea00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc77806bc00 0x7fc77806e0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009630 tx=0x7fc78c008030 comp rx=0 tx=0).stop 2024-03-28T01:06:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ccbb0 msgr2=0x7fc790161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7fc79ea00640 1 --2- 172.21.15.33:0/195730853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ccbb0 0x7fc790161c70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc79806ab70 tx=0x7fc798068b90 comp rx=0 tx=0).stop 2024-03-28T01:06:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8592800640 1 -- 172.21.15.33:0/1706435754 reap_dead start 2024-03-28T01:06:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 shutdown_connections 2024-03-28T01:06:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.794+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 >> 172.21.15.33:0/1706435754 conn(0x7f858c03a8a0 msgr2=0x7f858c038760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.795+0000 7fc79ca00640 1 -- 172.21.15.33:0/195730853 reap_dead start 2024-03-28T01:06:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.795+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 shutdown_connections 2024-03-28T01:06:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.795+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 >> 172.21.15.33:0/195730853 conn(0x7fc79003ea60 msgr2=0x7fc790004f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.795+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 shutdown_connections 2024-03-28T01:06:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.795+0000 7f8593200640 1 -- 172.21.15.33:0/1706435754 wait complete. 2024-03-28T01:06:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.795+0000 7f8a8d400640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7f8a64073f20 0x7f8a64076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.797+0000 7f8a8d400640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7f8a64073f20 0x7f8a64076340 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.797+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7cc0c5170 con 0x7fe7b4073e50 2024-03-28T01:06:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.797+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 shutdown_connections 2024-03-28T01:06:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.797+0000 7fc79ea00640 1 -- 172.21.15.33:0/195730853 wait complete. 2024-03-28T01:06:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.798+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 --> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc2a8005390 con 0x7fc290073ea0 2024-03-28T01:06:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.798+0000 7fe7c7400640 1 -- 172.21.15.33:0/1886665066 <== osd.3 v2:172.21.15.33:6800/2717396394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe7cc0c5170 con 0x7fe7b4073e50 2024-03-28T01:06:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.799+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7fe7b4073e50 msgr2=0x7fe7b4076270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.799+0000 7fe7da000640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7fe7b4073e50 0x7fe7b4076270 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.799+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe7b406bbd0 msgr2=0x7fe7b406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.799+0000 7fc2a3e00640 1 -- 172.21.15.33:0/3815660160 <== osd.1 v2:172.21.15.33:6816/4011280304 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc2a8005390 con 0x7fc290073ea0 2024-03-28T01:06:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.799+0000 7fe7da000640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe7b406bbd0 0x7fe7b406e090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0034000 tx=0x7fe7c002f000 comp rx=0 tx=0).stop 2024-03-28T01:06:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.799+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc038870 msgr2=0x7fe7cc0bf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.799+0000 7fe7da000640 1 --2- 172.21.15.33:0/1886665066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc038870 0x7fe7cc0bf010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c570 tx=0x7fe7c800ca30 comp rx=0 tx=0).stop 2024-03-28T01:06:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.800+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== osd.2 v2:172.21.15.33:6808/1101919847 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8a640769f0 con 0x7f8a64073f20 2024-03-28T01:06:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.800+0000 7fe7d3e00640 1 -- 172.21.15.33:0/1886665066 reap_dead start 2024-03-28T01:06:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 shutdown_connections 2024-03-28T01:06:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 >> 172.21.15.33:0/1886665066 conn(0x7fe7cc03ea60 msgr2=0x7fe7cc004f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 shutdown_connections 2024-03-28T01:06:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7fc290073ea0 msgr2=0x7fc2900762c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc2b5600640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7fc290073ea0 0x7fc2900762c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc72c002dd0 con 0x7fc730074200 2024-03-28T01:06:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc29006bc20 msgr2=0x7fc29006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc2b5600640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc29006bc20 0x7fc29006e0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc29c02b830 tx=0x7fc29c008030 comp rx=0 tx=0).stop 2024-03-28T01:06:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fe7da000640 1 -- 172.21.15.33:0/1886665066 wait complete. 2024-03-28T01:06:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803a370 msgr2=0x7fc2a80c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc2b5600640 1 --2- 172.21.15.33:0/3815660160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a803a370 0x7fc2a80c8850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b410 tx=0x7fc2a400b8d0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.801+0000 7fc2b4c00640 1 -- 172.21.15.33:0/3815660160 reap_dead start 2024-03-28T01:06:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.802+0000 7fc74cc00640 1 -- 172.21.15.33:0/776875024 <== osd.7 v2:172.21.15.194:6824/2849955959 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc72c002dd0 con 0x7fc730074200 2024-03-28T01:06:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.803+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 shutdown_connections 2024-03-28T01:06:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.803+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 >> 172.21.15.33:0/3815660160 conn(0x7fc2a803ec80 msgr2=0x7fc2a80ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.803+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7fc730074200 msgr2=0x7fc730076620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.803+0000 7fc756400640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7fc730074200 0x7fc730076620 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc73006c000 msgr2=0x7fc73006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc756400640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc73006c000 0x7fc73006e4c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0090b0 tx=0x7fc73c008030 comp rx=0 tx=0).stop 2024-03-28T01:06:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7480ccbb0 msgr2=0x7fc748036af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc756400640 1 --2- 172.21.15.33:0/776875024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7480ccbb0 0x7fc748036af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc750069ea0 tx=0x7fc750068b90 comp rx=0 tx=0).stop 2024-03-28T01:06:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 shutdown_connections 2024-03-28T01:06:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc755a00640 1 -- 172.21.15.33:0/776875024 reap_dead start 2024-03-28T01:06:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc2b5600640 1 -- 172.21.15.33:0/3815660160 wait complete. 2024-03-28T01:06:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 shutdown_connections 2024-03-28T01:06:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.804+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 >> 172.21.15.33:0/776875024 conn(0x7fc74803ea60 msgr2=0x7fc7480caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.805+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 shutdown_connections 2024-03-28T01:06:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.805+0000 7fc756400640 1 -- 172.21.15.33:0/776875024 wait complete. 2024-03-28T01:06:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.808+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 --> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f11e006b470 con 0x7f11c0073fc0 2024-03-28T01:06:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.809+0000 7f11dd600640 1 -- 172.21.15.33:0/3957892877 <== osd.5 v2:172.21.15.194:6816/3192443429 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f11e006b470 con 0x7f11c0073fc0 2024-03-28T01:06:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.811+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f11c0073fc0 msgr2=0x7f11c00763e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.811+0000 7f11e6c00640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f11c0073fc0 0x7f11c00763e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.813+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f11c006bdc0 msgr2=0x7f11c006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.813+0000 7f11e6c00640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f11c006bdc0 0x7f11c006e280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f11d8034000 tx=0x7f11d802f000 comp rx=0 tx=0).stop 2024-03-28T01:06:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.813+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0137860 msgr2=0x7f11e01e4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.813+0000 7f11e6c00640 1 --2- 172.21.15.33:0/3957892877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11e0137860 0x7f11e01e4350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d7a0 tx=0x7f11d000dc60 comp rx=0 tx=0).stop 2024-03-28T01:06:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.813+0000 7f11e6200640 1 -- 172.21.15.33:0/3957892877 reap_dead start 2024-03-28T01:06:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.820+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 shutdown_connections 2024-03-28T01:06:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.820+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 >> 172.21.15.33:0/3957892877 conn(0x7f11e00b89c0 msgr2=0x7f11e00b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.822+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 shutdown_connections 2024-03-28T01:06:29.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.822+0000 7f11e6c00640 1 -- 172.21.15.33:0/3957892877 wait complete. 2024-03-28T01:06:29.830 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:29.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-28T01:06:29.839 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2024-03-28T01:06:29.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T01:06:29.840 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:29.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.834+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a60002dd0 con 0x7f8a64073f20 2024-03-28T01:06:29.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.835+0000 7f8a84c00640 1 -- 172.21.15.33:0/50039831 <== osd.2 v2:172.21.15.33:6808/1101919847 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8a60002dd0 con 0x7f8a64073f20 2024-03-28T01:06:29.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7f8a64073f20 msgr2=0x7f8a64076340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7f8a64073f20 0x7f8a64076340 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T01:06:29.842 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-03-28T01:06:29.843 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:29.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T01:06:29.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8a6406bca0 msgr2=0x7f8a6406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8a6406bca0 0x7f8a6406e160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a7002b8e0 tx=0x7f8a70008030 comp rx=0 tx=0).stop 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a880b4780 msgr2=0x7f8a88133c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 --2- 172.21.15.33:0/50039831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a880b4780 0x7f8a88133c50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000e870 tx=0x7f8a8000ec70 comp rx=0 tx=0).stop 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8d400640 1 -- 172.21.15.33:0/50039831 reap_dead start 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 shutdown_connections 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 >> 172.21.15.33:0/50039831 conn(0x7f8a880b57b0 msgr2=0x7f8a880b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 shutdown_connections 2024-03-28T01:06:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.836+0000 7f8a8de00640 1 -- 172.21.15.33:0/50039831 wait complete. 2024-03-28T01:06:29.852 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:29.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-28T01:06:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.857+0000 7fdc10a00640 1 Processor -- start 2024-03-28T01:06:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.857+0000 7fdc10a00640 1 -- start start 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.858+0000 7fdc10a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c0b5830 0x7fdc0c0b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.858+0000 7fdc10a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdc0c140f50 0x7fdc0c1413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.858+0000 7fdc10a00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0c1418f0 con 0x7fdc0c14a830 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.858+0000 7fdc10a00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0c0b6a90 con 0x7fdc0c140f50 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.858+0000 7fdc10a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0b5c10 con 0x7fdc0c0b5830 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.858+0000 7fdc09400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdc0c140f50 0x7fdc0c1413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.858+0000 7fdc09400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdc0c140f50 0x7fdc0c1413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36414/0 (socket says 172.21.15.33:36414) 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.859+0000 7fdc09400640 1 -- 172.21.15.33:0/1097197342 learned_addr learned my addr 172.21.15.33:0/1097197342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.859+0000 7fdc09e00640 1 --2- 172.21.15.33:0/1097197342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c0b5830 0x7fdc0c0b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.859+0000 7fdc09400640 1 -- 172.21.15.33:0/1097197342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c0b5830 msgr2=0x7fdc0c0b9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.859+0000 7fdc09400640 1 --2- 172.21.15.33:0/1097197342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c0b5830 0x7fdc0c0b9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.859+0000 7fdc09400640 1 -- 172.21.15.33:0/1097197342 >> v1:172.21.15.194:6789/0 conn(0x7fdc0c14a830 legacy=0x7fdc0c14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.859+0000 7fdc09400640 1 -- 172.21.15.33:0/1097197342 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0b4290 con 0x7fdc0c140f50 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.861+0000 7fdc09400640 1 --2- 172.21.15.33:0/1097197342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c140f50 0x7fdc0c1413b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009870 tx=0x7fdbf802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39472632f71de1cb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.861+0000 7fdc08a00640 1 -- 172.21.15.33:0/1097197342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8035020 con 0x7fdc0c140f50 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.862+0000 7fdc08a00640 1 -- 172.21.15.33:0/1097197342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf8031640 con 0x7fdc0c140f50 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.862+0000 7fdc08a00640 1 -- 172.21.15.33:0/1097197342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf802d040 con 0x7fdc0c140f50 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.862+0000 7fdc10a00640 1 -- 172.21.15.33:0/1097197342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c140f50 msgr2=0x7fdc0c1413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.862+0000 7fdc10a00640 1 --2- 172.21.15.33:0/1097197342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c140f50 0x7fdc0c1413b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009870 tx=0x7fdbf802b8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.863+0000 7fdc10a00640 1 -- 172.21.15.33:0/1097197342 shutdown_connections 2024-03-28T01:06:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.863+0000 7fdc10a00640 1 --2- 172.21.15.33:0/1097197342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c140f50 0x7fdc0c1413b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.863+0000 7fdc10a00640 1 --2- 172.21.15.33:0/1097197342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c0b5830 0x7fdc0c0b9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.863+0000 7fdc10a00640 1 -- 172.21.15.33:0/1097197342 >> 172.21.15.33:0/1097197342 conn(0x7fdc0c072b80 msgr2=0x7fdc0c072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.863+0000 7fdc10a00640 1 -- 172.21.15.33:0/1097197342 shutdown_connections 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.864+0000 7fdc10a00640 1 -- 172.21.15.33:0/1097197342 wait complete. 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.866+0000 7fdc10a00640 1 Processor -- start 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.866+0000 7fdc10a00640 1 -- start start 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.867+0000 7fdc10a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c0b5830 0x7fdc0c139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.867+0000 7fdc10a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc0c140f50 0x7fdc0c1dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.867+0000 7fdc10a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c14a830 0x7fdc0c13c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.867+0000 7fdc10a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0ba280 con 0x7fdc0c0b5830 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.867+0000 7fdc10a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0b4850 con 0x7fdc0c140f50 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.867+0000 7fdc10a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0ba400 con 0x7fdc0c14a830 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c0b5830 0x7fdc0c139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c0b5830 0x7fdc0c139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36416/0 (socket says 172.21.15.33:36416) 2024-03-28T01:06:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc09e00640 1 -- 172.21.15.33:0/1148838374 learned_addr learned my addr 172.21.15.33:0/1148838374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc0a800640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c14a830 0x7fdc0c13c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc09e00640 1 -- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c14a830 msgr2=0x7fdc0c13c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc09e00640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc0c14a830 0x7fdc0c13c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc09400640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc0c140f50 0x7fdc0c1dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.868+0000 7fdc09e00640 1 -- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc0c140f50 msgr2=0x7fdc0c1dccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T01:06:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.869+0000 7fdc09e00640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc0c140f50 0x7fdc0c1dccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.869+0000 7fdc09e00640 1 -- 172.21.15.33:0/1148838374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c13bd30 con 0x7fdc0c0b5830 2024-03-28T01:06:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.869+0000 7fdc09400640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc0c140f50 0x7fdc0c1dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.869+0000 7fdc09e00640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c0b5830 0x7fdc0c139ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00d800 tx=0x7fdbfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.871+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00b6c0 con 0x7fdc0c0b5830 2024-03-28T01:06:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.871+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc019070 con 0x7fdc0c0b5830 2024-03-28T01:06:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.871+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00bac0 con 0x7fdc0c0b5830 2024-03-28T01:06:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.872+0000 7fdc10a00640 1 -- 172.21.15.33:0/1148838374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c1d20e0 con 0x7fdc0c0b5830 2024-03-28T01:06:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.872+0000 7fdc10a00640 1 -- 172.21.15.33:0/1148838374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c1dd230 con 0x7fdc0c0b5830 2024-03-28T01:06:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.873+0000 7fdc10a00640 1 -- 172.21.15.33:0/1148838374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc0c1d1ed0 con 0x7fdc0c0b5830 2024-03-28T01:06:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.875+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdbfc00bc60 con 0x7fdc0c0b5830 2024-03-28T01:06:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.876+0000 7fdc02000640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdbec06bc00 0x7fdbec06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.876+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbfc0573a0 con 0x7fdc0c0b5830 2024-03-28T01:06:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.877+0000 7fdc02000640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7fdbec0749a0 0x7fdbec076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.877+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 --> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbec077510 con 0x7fdbec0749a0 2024-03-28T01:06:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.877+0000 7fdc09400640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdbec06bc00 0x7fdbec06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.877+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbfc055d30 con 0x7fdc0c0b5830 2024-03-28T01:06:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.878+0000 7fdc09400640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdbec06bc00 0x7fdbec06e0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8036000 tx=0x7fdbf8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.879+0000 7fdc0a800640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7fdbec0749a0 0x7fdbec076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:29.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.885+0000 7fdc0a800640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7fdbec0749a0 0x7fdbec076e40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.889+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== osd.6 v2:172.21.15.194:6800/1743959830 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdbec077510 con 0x7fdbec0749a0 2024-03-28T01:06:29.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.926+0000 7fdc10a00640 1 -- 172.21.15.33:0/1148838374 --> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdc0c075380 con 0x7fdbec0749a0 2024-03-28T01:06:29.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.929+0000 7fdc02000640 1 -- 172.21.15.33:0/1148838374 <== osd.6 v2:172.21.15.194:6800/1743959830 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdc0c075380 con 0x7fdbec0749a0 2024-03-28T01:06:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.930+0000 7fdbf7400640 1 -- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7fdbec0749a0 msgr2=0x7fdbec076e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.930+0000 7fdbf7400640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7fdbec0749a0 0x7fdbec076e40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.930+0000 7fdbf7400640 1 -- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdbec06bc00 msgr2=0x7fdbec06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.930+0000 7fdbf7400640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdbec06bc00 0x7fdbec06e0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8036000 tx=0x7fdbf8008030 comp rx=0 tx=0).stop 2024-03-28T01:06:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.931+0000 7fdbf7400640 1 -- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c0b5830 msgr2=0x7fdc0c139ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.931+0000 7fdbf7400640 1 --2- 172.21.15.33:0/1148838374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0c0b5830 0x7fdc0c139ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00d800 tx=0x7fdbfc00dcc0 comp rx=0 tx=0).stop 2024-03-28T01:06:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.931+0000 7fdc0a800640 1 -- 172.21.15.33:0/1148838374 reap_dead start 2024-03-28T01:06:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.931+0000 7fdbf7400640 1 -- 172.21.15.33:0/1148838374 shutdown_connections 2024-03-28T01:06:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.931+0000 7fdbf7400640 1 -- 172.21.15.33:0/1148838374 >> 172.21.15.33:0/1148838374 conn(0x7fdc0c072b80 msgr2=0x7fdc0c0b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.934+0000 7fdbf7400640 1 -- 172.21.15.33:0/1148838374 shutdown_connections 2024-03-28T01:06:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:29.934+0000 7fdbf7400640 1 -- 172.21.15.33:0/1148838374 wait complete. 2024-03-28T01:06:29.963 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2024-03-28T01:06:29.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-28T01:06:30.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.121+0000 7f4b0f200640 1 Processor -- start 2024-03-28T01:06:30.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.121+0000 7f4b0f200640 1 -- start start 2024-03-28T01:06:30.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.126+0000 7f4b0f200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f4b0003ae20 0x7f4b000c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.126+0000 7f4b0f200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0003b280 con 0x7f4b000c6740 2024-03-28T01:06:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.126+0000 7f4b0f200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b000cc730 con 0x7f4b0003a370 2024-03-28T01:06:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.126+0000 7f4b0f200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b000cca60 con 0x7f4b0003ae20 2024-03-28T01:06:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.127+0000 7f4b07400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f4b0003ae20 0x7f4b000c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.127+0000 7f4b07e00640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7f4b0003a370 0x7f4b000ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37896/0 (socket says 172.21.15.33:37896) 2024-03-28T01:06:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.127+0000 7f4b07e00640 1 -- 172.21.15.33:0/3685015277 learned_addr learned my addr 172.21.15.33:0/3685015277 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.128+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689472156 0 0) 0x7f4b0003b280 con 0x7f4b000c6740 2024-03-28T01:06:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.128+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af0003520 con 0x7f4b000c6740 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755208938 0 0) 0x7f4af0003520 con 0x7f4b000c6740 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 >> v2:172.21.15.194:3300/0 conn(0x7f4b0003ae20 msgr2=0x7f4b000c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 --2- 172.21.15.33:0/3685015277 >> v2:172.21.15.194:3300/0 conn(0x7f4b0003ae20 0x7f4b000c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 >> v1:172.21.15.194:6789/0 conn(0x7f4b0003a370 legacy=0x7f4b000ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0003a750 con 0x7f4b000c6740 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f4b0804fc50 con 0x7f4b000c6740 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 >> v1:172.21.15.33:6789/0 conn(0x7f4b000c6740 legacy=0x7f4b000d6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4af00039b0 0x7f4af0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4af00051c0 0x7f4af00141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000cca60 con 0x7f4af00039b0 2024-03-28T01:06:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0014900 con 0x7f4af0004800 2024-03-28T01:06:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.129+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0014b00 con 0x7f4af00051c0 2024-03-28T01:06:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.131+0000 7f4b07400640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.131+0000 7f4b07e00640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4af00039b0 0x7f4af0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.131+0000 7f4b0d200640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4af00051c0 0x7f4af00141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.132+0000 7f4b07e00640 1 -- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4af00051c0 msgr2=0x7f4af00141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.132+0000 7f4b07e00640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4af00051c0 0x7f4af00141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.132+0000 7f4b07e00640 1 -- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af0004800 msgr2=0x7f4af0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.132+0000 7f4b07e00640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.132+0000 7f4b07e00640 1 -- 172.21.15.33:0/3685015277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0003a750 con 0x7f4af00039b0 2024-03-28T01:06:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.134+0000 7f4b0d200640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4af00051c0 0x7f4af00141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.134+0000 7f4b0d200640 1 -- 172.21.15.33:0/3685015277 reap_dead start 2024-03-28T01:06:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.135+0000 7f4b07e00640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4af00039b0 0x7f4af0003d90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4afc001930 tx=0x7f4af0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b19fef96322cc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.135+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc037070 con 0x7f4af00039b0 2024-03-28T01:06:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.136+0000 7f4b06a00640 1 -- 172.21.15.33:0/3685015277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc02d040 con 0x7f4af00039b0 2024-03-28T01:06:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.137+0000 7f4b0f200640 1 -- 172.21.15.33:0/3685015277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4af00039b0 msgr2=0x7f4af0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.137+0000 7f4b0f200640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4af00039b0 0x7f4af0003d90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4afc001930 tx=0x7f4af0005830 comp rx=0 tx=0).stop 2024-03-28T01:06:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.137+0000 7f4b0f200640 1 -- 172.21.15.33:0/3685015277 shutdown_connections 2024-03-28T01:06:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.138+0000 7f4b0f200640 1 --2- 172.21.15.33:0/3685015277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4af00039b0 0x7f4af0003d90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.138+0000 7f4b0f200640 1 -- 172.21.15.33:0/3685015277 >> 172.21.15.33:0/3685015277 conn(0x7f4b0003ec80 msgr2=0x7f4b0003f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.138+0000 7f4b0f200640 1 -- 172.21.15.33:0/3685015277 shutdown_connections 2024-03-28T01:06:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.138+0000 7f4b0f200640 1 -- 172.21.15.33:0/3685015277 wait complete. 2024-03-28T01:06:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.142+0000 7f4b0f200640 1 Processor -- start 2024-03-28T01:06:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.142+0000 7f4b0f200640 1 -- start start 2024-03-28T01:06:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.144+0000 7f4b0f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af00039b0 0x7f4b000c2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.144+0000 7f4b0f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b000ce940 0x7f4b000c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.145+0000 7f4b07e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af00039b0 0x7f4b000c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.145+0000 7f4b07e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af00039b0 0x7f4b000c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44418/0 (socket says 172.21.15.33:44418) 2024-03-28T01:06:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.145+0000 7f4b07400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b000ce940 0x7f4b000c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.145+0000 7f4b07400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b000ce940 0x7f4b000c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36426/0 (socket says 172.21.15.33:36426) 2024-03-28T01:06:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.146+0000 7f4b07e00640 1 -- 172.21.15.33:0/1345102249 learned_addr learned my addr 172.21.15.33:0/1345102249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.146+0000 7f4b0f200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b001614d0 0x7f4b000c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.146+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0003bed0 con 0x7f4b000ce940 2024-03-28T01:06:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.146+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000c6bb0 con 0x7f4af00039b0 2024-03-28T01:06:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.146+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00039210 con 0x7f4b001614d0 2024-03-28T01:06:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.146+0000 7f4b0d200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b001614d0 0x7f4b000c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.148+0000 7f4b07e00640 1 -- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b001614d0 msgr2=0x7f4b000c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.148+0000 7f4b07e00640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b001614d0 0x7f4b000c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.148+0000 7f4b07e00640 1 -- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b000ce940 msgr2=0x7f4b000c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.148+0000 7f4b07e00640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b000ce940 0x7f4b000c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.148+0000 7f4b07e00640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00160d10 con 0x7f4af00039b0 2024-03-28T01:06:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.150+0000 7f4b0d200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b001614d0 0x7f4b000c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.150+0000 7f4b07400640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b000ce940 0x7f4b000c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.153+0000 7f4b07e00640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af00039b0 0x7f4b000c2460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4afc02bf10 tx=0x7f4afc001230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.157+0000 7f4afbe00640 1 -- 172.21.15.33:0/1345102249 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc037070 con 0x7f4af00039b0 2024-03-28T01:06:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.157+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b000c3dc0 con 0x7f4af00039b0 2024-03-28T01:06:30.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.157+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b00161960 con 0x7f4af00039b0 2024-03-28T01:06:30.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.157+0000 7f4afbe00640 1 -- 172.21.15.33:0/1345102249 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc031630 con 0x7f4af00039b0 2024-03-28T01:06:30.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.157+0000 7f4afbe00640 1 -- 172.21.15.33:0/1345102249 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc037070 con 0x7f4af00039b0 2024-03-28T01:06:30.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.159+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b00005390 con 0x7f4af00039b0 2024-03-28T01:06:30.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.166+0000 7f4afbe00640 1 -- 172.21.15.33:0/1345102249 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4afc0317e0 con 0x7f4af00039b0 2024-03-28T01:06:30.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.166+0000 7fdde7200640 1 Processor -- start 2024-03-28T01:06:30.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.167+0000 7fdde7200640 1 -- start start 2024-03-28T01:06:30.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.167+0000 7f4afbe00640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f4ae806be00 0x7f4ae806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.167+0000 7f4afbe00640 1 -- 172.21.15.33:0/1345102249 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4afc0ab810 con 0x7f4af00039b0 2024-03-28T01:06:30.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.168+0000 7f4b07400640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f4ae806be00 0x7f4ae806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.168+0000 7ffabe200640 1 Processor -- start 2024-03-28T01:06:30.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.168+0000 7ffabe200640 1 -- start start 2024-03-28T01:06:30.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.169+0000 7fdde7200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fdde0035600 0x7fdde00365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.169+0000 7fdde7200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdde00359e0 0x7fdde00337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.169+0000 7f4b07400640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f4ae806be00 0x7f4ae806e2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b000ced20 tx=0x7f4af4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.170+0000 7fdde5e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fdde0035600 0x7fdde00365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.170+0000 7fdde6800640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdde0033d10 0x7fdde00c85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41084/0 (socket says 172.21.15.33:41084) 2024-03-28T01:06:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.170+0000 7fdde6800640 1 -- 172.21.15.33:0/4001175356 learned_addr learned my addr 172.21.15.33:0/4001175356 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.170+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde0029d80 con 0x7fdde0033d10 2024-03-28T01:06:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.170+0000 7fdde5400640 1 --2- 172.21.15.33:0/4001175356 >> v2:172.21.15.33:3300/0 conn(0x7fdde00359e0 0x7fdde00337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.170+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde0029ff0 con 0x7fdde00359e0 2024-03-28T01:06:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.170+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde0035e60 con 0x7fdde0035600 2024-03-28T01:06:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7ffab7400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffab8147600 0x7ffab8148790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41100/0 (socket says 172.21.15.33:41100) 2024-03-28T01:06:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7ffab7400640 1 -- 172.21.15.33:0/2049600018 learned_addr learned my addr 172.21.15.33:0/2049600018 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7fdde4a00640 1 -- 172.21.15.33:0/4001175356 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272367477 0 0) 0x7fdde0029d80 con 0x7fdde0033d10 2024-03-28T01:06:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7ffabe200640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 0x7ffab8153340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7fdde4a00640 1 -- 172.21.15.33:0/4001175356 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc003520 con 0x7fdde0033d10 2024-03-28T01:06:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7fdde5400640 1 -- 172.21.15.33:0/4001175356 >> v2:172.21.15.194:3300/0 conn(0x7fdde0035600 msgr2=0x7fdde00365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7fdde5400640 1 --2- 172.21.15.33:0/4001175356 >> v2:172.21.15.194:3300/0 conn(0x7fdde0035600 0x7fdde00365a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7fdde5400640 1 -- 172.21.15.33:0/4001175356 >> v1:172.21.15.33:6789/0 conn(0x7fdde0033d10 legacy=0x7fdde00c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7fdde5400640 1 -- 172.21.15.33:0/4001175356 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0036100 con 0x7fdde00359e0 2024-03-28T01:06:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7fdde5e00640 1 --2- 172.21.15.33:0/4001175356 >> v2:172.21.15.194:3300/0 conn(0x7fdde0035600 0x7fdde00365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab80af9a0 con 0x7ffab8147600 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab80b2fa0 con 0x7ffab80b26b0 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.171+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab80afc10 con 0x7ffab80b2b10 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.172+0000 7ffab7e00640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 0x7ffab8153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.172+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4544546 0 0) 0x7ffab80af9a0 con 0x7ffab8147600 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.172+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c003520 con 0x7ffab8147600 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.172+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673696691 0 0) 0x7ffa9c003520 con 0x7ffab8147600 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.172+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 msgr2=0x7ffab8153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.172+0000 7ffab6000640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 0x7ffab8153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 >> v1:172.21.15.194:6789/0 conn(0x7ffab80b26b0 legacy=0x7ffab814fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80b0070 con 0x7ffab8147600 2024-03-28T01:06:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7ffaa8001670 con 0x7ffab8147600 2024-03-28T01:06:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 >> v1:172.21.15.33:6789/0 conn(0x7ffab8147600 legacy=0x7ffab8148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa9c0039e0 0x7ffa9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 0x7ffa9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa9c004d40 0x7ffa9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7f2bd3200640 1 Processor -- start 2024-03-28T01:06:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab80afc10 con 0x7ffa9c0039e0 2024-03-28T01:06:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c00bb50 con 0x7ffa9c004d40 2024-03-28T01:06:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7fdde5400640 1 --2- 172.21.15.33:0/4001175356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00359e0 0x7fdde00337d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdde0007730 tx=0x7fddd402df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d17c56752f84c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7fdde4a00640 1 -- 172.21.15.33:0/4001175356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd403b070 con 0x7fdde00359e0 2024-03-28T01:06:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7fdde4a00640 1 -- 172.21.15.33:0/4001175356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd402eae0 con 0x7fdde00359e0 2024-03-28T01:06:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c00bd50 con 0x7ffab80b2b10 2024-03-28T01:06:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.173+0000 7ffab7400640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa9c0039e0 0x7ffa9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7ffab7e00640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa9c004d40 0x7ffa9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7ffab6a00640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 0x7ffa9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7f2bd3200640 1 -- start start 2024-03-28T01:06:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7fdde4a00640 1 -- 172.21.15.33:0/4001175356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4036440 con 0x7fdde00359e0 2024-03-28T01:06:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.175+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00359e0 msgr2=0x7fdde00337d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.175+0000 7fdde7200640 1 --2- 172.21.15.33:0/4001175356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00359e0 0x7fdde00337d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdde0007730 tx=0x7fddd402df40 comp rx=0 tx=0).stop 2024-03-28T01:06:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7ffab7400640 1 -- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 msgr2=0x7ffa9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7ffab7400640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab80b2b10 0x7ffa9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7ffab7400640 1 -- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa9c004d40 msgr2=0x7ffa9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7ffab7400640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa9c004d40 0x7ffa9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.174+0000 7ffab7400640 1 -- 172.21.15.33:0/2049600018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c003520 con 0x7ffa9c0039e0 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.175+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 shutdown_connections 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.175+0000 7fdde7200640 1 --2- 172.21.15.33:0/4001175356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00359e0 0x7fdde00337d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.175+0000 7fdde7200640 1 --2- 172.21.15.33:0/4001175356 >> v2:172.21.15.194:3300/0 conn(0x7fdde0035600 0x7fdde00365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.175+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 >> 172.21.15.33:0/4001175356 conn(0x7fdde003a8a0 msgr2=0x7fdde003acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.175+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 shutdown_connections 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.176+0000 7fdde7200640 1 -- 172.21.15.33:0/4001175356 wait complete. 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.176+0000 7ffab7400640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa9c0039e0 0x7ffa9c003de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800c160 tx=0x7ffa9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd051025babcec10 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.176+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8007c80 con 0x7ffa9c0039e0 2024-03-28T01:06:30.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.176+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8030070 con 0x7ffa9c0039e0 2024-03-28T01:06:30.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.176+0000 7ffab6000640 1 -- 172.21.15.33:0/2049600018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8005d70 con 0x7ffa9c0039e0 2024-03-28T01:06:30.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.176+0000 7f2bd3200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f2bcc0365a0 0x7f2bcc035600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.177+0000 7f2bd3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.177+0000 7f2bd3200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc029d80 con 0x7f2bcc006ea0 2024-03-28T01:06:30.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.177+0000 7f2bd3200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bcc036a20 con 0x7f2bcc0365a0 2024-03-28T01:06:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.177+0000 7f2bd3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc029ff0 con 0x7f2bcc035b40 2024-03-28T01:06:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.177+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa9c0039e0 msgr2=0x7ffa9c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.177+0000 7ffabe200640 1 --2- 172.21.15.33:0/2049600018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa9c0039e0 0x7ffa9c003de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800c160 tx=0x7ffa9c005830 comp rx=0 tx=0).stop 2024-03-28T01:06:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.178+0000 7ffab7e00640 1 -- 172.21.15.33:0/2049600018 reap_dead start 2024-03-28T01:06:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.177+0000 7f2bd2800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.178+0000 7f2bd1e00640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7f2bcc006ea0 0x7f2bcc0337d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37904/0 (socket says 172.21.15.33:37904) 2024-03-28T01:06:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.178+0000 7f2bd1e00640 1 -- 172.21.15.33:0/10302181 learned_addr learned my addr 172.21.15.33:0/10302181 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.178+0000 7f2bd1400640 1 --2- 172.21.15.33:0/10302181 >> v2:172.21.15.194:3300/0 conn(0x7f2bcc0365a0 0x7f2bcc035600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.178+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 shutdown_connections 2024-03-28T01:06:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.178+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 >> 172.21.15.33:0/2049600018 conn(0x7ffab80b69d0 msgr2=0x7ffab80b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.178+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 shutdown_connections 2024-03-28T01:06:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.179+0000 7f7131a00640 1 Processor -- start 2024-03-28T01:06:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.179+0000 7ffabe200640 1 -- 172.21.15.33:0/2049600018 wait complete. 2024-03-28T01:06:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.179+0000 7f2bd2800640 1 -- 172.21.15.33:0/10302181 >> v2:172.21.15.194:3300/0 conn(0x7f2bcc0365a0 msgr2=0x7f2bcc035600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.179+0000 7f2bd2800640 1 --2- 172.21.15.33:0/10302181 >> v2:172.21.15.194:3300/0 conn(0x7f2bcc0365a0 0x7f2bcc035600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.179+0000 7f2bd2800640 1 -- 172.21.15.33:0/10302181 >> v1:172.21.15.194:6789/0 conn(0x7f2bcc006ea0 legacy=0x7f2bcc0337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.179+0000 7f2bd2800640 1 -- 172.21.15.33:0/10302181 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc037020 con 0x7f2bcc035b40 2024-03-28T01:06:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.180+0000 7fdde7200640 1 Processor -- start 2024-03-28T01:06:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.180+0000 7fdde7200640 1 -- start start 2024-03-28T01:06:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7ffabe200640 1 Processor -- start 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7ffabe200640 1 -- start start 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.179+0000 7f7131a00640 1 -- start start 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.180+0000 7f7131a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f712c0b0410 0x7f712c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.180+0000 7f7131a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b4880 0x7f712c14f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7f7131a00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c0b2610 con 0x7f712c0b44a0 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7f7131a00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c06c840 con 0x7f712c0b0410 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7f7131a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0b1bc0 con 0x7f712c0b4880 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7f712a000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f712c0b0410 0x7f712c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7f712a000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f712c0b0410 0x7f712c14f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36458/0 (socket says 172.21.15.33:36458) 2024-03-28T01:06:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7f712a000640 1 -- 172.21.15.33:0/141741546 learned_addr learned my addr 172.21.15.33:0/141741546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7fdde7200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdde0033d10 0x7fdde00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7fdde7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0035600 0x7fdde00c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7fdde7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdde00359e0 0x7fdde00329a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7f712b400640 1 --2- 172.21.15.33:0/141741546 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b4880 0x7f712c14f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7f2bd2800640 1 --2- 172.21.15.33:0/10302181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c85c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40097a0 tx=0x7f2bc402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44aed51b5197b7da server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7fdde7200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00c8e50 con 0x7fdde0035600 2024-03-28T01:06:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7f712b400640 1 -- 172.21.15.33:0/141741546 >> v2:172.21.15.33:3300/0 conn(0x7f712c0b0410 msgr2=0x7f712c14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7f712b400640 1 --2- 172.21.15.33:0/141741546 >> v2:172.21.15.33:3300/0 conn(0x7f712c0b0410 0x7f712c14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7f712b400640 1 -- 172.21.15.33:0/141741546 >> v1:172.21.15.33:6789/0 conn(0x7f712c0b44a0 legacy=0x7f712c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7f712b400640 1 -- 172.21.15.33:0/141741546 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0b1ea0 con 0x7f712c0b4880 2024-03-28T01:06:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7f2bd0a00640 1 -- 172.21.15.33:0/10302181 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4035020 con 0x7f2bcc035b40 2024-03-28T01:06:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7fdde7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00c8cd0 con 0x7fdde00359e0 2024-03-28T01:06:30.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7ffabe200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffab8149f00 0x7ffab81458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7ffabe200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffab81469e0 0x7ffab81dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7ffabe200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab8146e20 0x7ffab81dd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7ffabe200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8072f00 con 0x7ffab81469e0 2024-03-28T01:06:30.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7ffabe200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8147e00 con 0x7ffab8149f00 2024-03-28T01:06:30.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7ffabe200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab80ae8d0 con 0x7ffab8146e20 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.181+0000 7fdde7200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0006ea0 con 0x7fdde0033d10 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.182+0000 7fdde5400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0035600 0x7fdde00c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde5400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0035600 0x7fdde00c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36474/0 (socket says 172.21.15.33:36474) 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde5400640 1 -- 172.21.15.33:0/3163573173 learned_addr learned my addr 172.21.15.33:0/3163573173 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde6800640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdde00359e0 0x7fdde00329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde5400640 1 -- 172.21.15.33:0/3163573173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdde0033d10 msgr2=0x7fdde00c4af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde5400640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdde0033d10 0x7fdde00c4af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde5400640 1 -- 172.21.15.33:0/3163573173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdde00359e0 msgr2=0x7fdde00329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde5400640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdde00359e0 0x7fdde00329a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7fdde5400640 1 -- 172.21.15.33:0/3163573173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00cb300 con 0x7fdde0035600 2024-03-28T01:06:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7f2bd0a00640 1 -- 172.21.15.33:0/10302181 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4031830 con 0x7f2bcc035b40 2024-03-28T01:06:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.183+0000 7f2bd0a00640 1 -- 172.21.15.33:0/10302181 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4031c40 con 0x7f2bcc035b40 2024-03-28T01:06:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7f2bd3200640 1 -- 172.21.15.33:0/10302181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 msgr2=0x7f2bcc0c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7f2bd3200640 1 --2- 172.21.15.33:0/10302181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c85c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40097a0 tx=0x7f2bc402b8d0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7ffab6a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffab81469e0 0x7ffab81dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7ffab7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffab8149f00 0x7ffab81458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7ffab7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffab8149f00 0x7ffab81458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44472/0 (socket says 172.21.15.33:44472) 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7ffab7400640 1 -- 172.21.15.33:0/3517054388 learned_addr learned my addr 172.21.15.33:0/3517054388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7ffab7e00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab8146e20 0x7ffab81dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7f2bd3200640 1 -- 172.21.15.33:0/10302181 shutdown_connections 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7f2bd3200640 1 --2- 172.21.15.33:0/10302181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c85c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7f2bd3200640 1 --2- 172.21.15.33:0/10302181 >> v2:172.21.15.194:3300/0 conn(0x7f2bcc0365a0 0x7f2bcc035600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7f2bd3200640 1 -- 172.21.15.33:0/10302181 >> 172.21.15.33:0/10302181 conn(0x7f2bcc03a8a0 msgr2=0x7f2bcc03acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.184+0000 7f2bd3200640 1 -- 172.21.15.33:0/10302181 shutdown_connections 2024-03-28T01:06:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7f712b400640 1 --2- 172.21.15.33:0/141741546 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b4880 0x7f712c14f7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f712c06c9c0 tx=0x7f712402bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d24c78d7ebfd96b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7ffab6a00640 1 -- 172.21.15.33:0/3517054388 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab8146e20 msgr2=0x7ffab81dd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7ffab6a00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab8146e20 0x7ffab81dd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7ffab6a00640 1 -- 172.21.15.33:0/3517054388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffab8149f00 msgr2=0x7ffab81458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7ffab6a00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffab8149f00 0x7ffab81458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7ffab6a00640 1 -- 172.21.15.33:0/3517054388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab8146250 con 0x7ffab81469e0 2024-03-28T01:06:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7fdde5400640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0035600 0x7fdde00c9780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fddd402e550 tx=0x7fddd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7ffab6a00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffab81469e0 0x7ffab81dcb10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4001b00 tx=0x7ffaa40164a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7fddde000640 1 -- 172.21.15.33:0/3163573173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd403b040 con 0x7fdde0035600 2024-03-28T01:06:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7fddde000640 1 -- 172.21.15.33:0/3163573173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4041430 con 0x7fdde0035600 2024-03-28T01:06:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7fddde000640 1 -- 172.21.15.33:0/3163573173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4041800 con 0x7fdde0035600 2024-03-28T01:06:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde00cb5e0 con 0x7fdde0035600 2024-03-28T01:06:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7ffaaf400640 1 -- 172.21.15.33:0/3517054388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4021070 con 0x7ffab81469e0 2024-03-28T01:06:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7ffaaf400640 1 -- 172.21.15.33:0/3517054388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa4016d70 con 0x7ffab81469e0 2024-03-28T01:06:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7ffaaf400640 1 -- 172.21.15.33:0/3517054388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4005310 con 0x7ffab81469e0 2024-03-28T01:06:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.186+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde002b130 con 0x7fdde0035600 2024-03-28T01:06:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.187+0000 7f7129600640 1 -- 172.21.15.33:0/141741546 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124039070 con 0x7f712c0b4880 2024-03-28T01:06:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.187+0000 7f7129600640 1 -- 172.21.15.33:0/141741546 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712402cab0 con 0x7f712c0b4880 2024-03-28T01:06:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.187+0000 7f7129600640 1 -- 172.21.15.33:0/141741546 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124034440 con 0x7f712c0b4880 2024-03-28T01:06:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.185+0000 7f2bd3200640 1 -- 172.21.15.33:0/10302181 wait complete. 2024-03-28T01:06:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.187+0000 7ffabe200640 1 -- 172.21.15.33:0/3517054388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab814a2e0 con 0x7ffab81469e0 2024-03-28T01:06:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.187+0000 7ffabe200640 1 -- 172.21.15.33:0/3517054388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab8142a60 con 0x7ffab81469e0 2024-03-28T01:06:30.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f2bd3200640 1 Processor -- start 2024-03-28T01:06:30.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f7131a00640 1 -- 172.21.15.33:0/141741546 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b4880 msgr2=0x7f712c14f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f7131a00640 1 --2- 172.21.15.33:0/141741546 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b4880 0x7f712c14f7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f712c06c9c0 tx=0x7f712402bd00 comp rx=0 tx=0).stop 2024-03-28T01:06:30.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f7131a00640 1 -- 172.21.15.33:0/141741546 shutdown_connections 2024-03-28T01:06:30.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f7131a00640 1 --2- 172.21.15.33:0/141741546 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b4880 0x7f712c14f7b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f7131a00640 1 --2- 172.21.15.33:0/141741546 >> v2:172.21.15.33:3300/0 conn(0x7f712c0b0410 0x7f712c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f7131a00640 1 -- 172.21.15.33:0/141741546 >> 172.21.15.33:0/141741546 conn(0x7f712c0b5740 msgr2=0x7f712c0b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.188+0000 7f2bd3200640 1 -- start start 2024-03-28T01:06:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f2bd3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bcc006ea0 0x7f2bcc0a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f2bd3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f2bd3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bcc0365a0 0x7f2bcc0a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f7131a00640 1 -- 172.21.15.33:0/141741546 shutdown_connections 2024-03-28T01:06:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f2bd3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0c2d70 con 0x7f2bcc0365a0 2024-03-28T01:06:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f2bd3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0c2bf0 con 0x7f2bcc006ea0 2024-03-28T01:06:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f2bd3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0c8b00 con 0x7f2bcc035b40 2024-03-28T01:06:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7f7131a00640 1 -- 172.21.15.33:0/141741546 wait complete. 2024-03-28T01:06:30.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.190+0000 7f2bd1e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bcc006ea0 0x7f2bcc0a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.190+0000 7f2bd1e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bcc006ea0 0x7f2bcc0a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44484/0 (socket says 172.21.15.33:44484) 2024-03-28T01:06:30.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.190+0000 7f2bd1e00640 1 -- 172.21.15.33:0/2259042880 learned_addr learned my addr 172.21.15.33:0/2259042880 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.190+0000 7f2bd2800640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bcc0365a0 0x7f2bcc0a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.190+0000 7f2bd1400640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.191+0000 7f2bd1e00640 1 -- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 msgr2=0x7f2bcc0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.191+0000 7f2bd1e00640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.191+0000 7f2bd1e00640 1 -- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bcc0365a0 msgr2=0x7f2bcc0a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.191+0000 7f2bd1e00640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bcc0365a0 0x7f2bcc0a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.191+0000 7f2bd1e00640 1 -- 172.21.15.33:0/2259042880 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0a8a40 con 0x7f2bcc006ea0 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.191+0000 7f2bd2800640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bcc0365a0 0x7f2bcc0a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.191+0000 7f2bd1e00640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bcc006ea0 0x7f2bcc0a6d20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4069ea0 tx=0x7f2bd4068b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.189+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde00c7d30 con 0x7fdde0035600 2024-03-28T01:06:30.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.192+0000 7fdec8000640 1 Processor -- start 2024-03-28T01:06:30.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.192+0000 7fdec8000640 1 -- start start 2024-03-28T01:06:30.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.192+0000 7f2bca000640 1 -- 172.21.15.33:0/2259042880 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4071900 con 0x7f2bcc006ea0 2024-03-28T01:06:30.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.192+0000 7f2bca000640 1 -- 172.21.15.33:0/2259042880 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4071ac0 con 0x7f2bcc006ea0 2024-03-28T01:06:30.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.192+0000 7f2bca000640 1 -- 172.21.15.33:0/2259042880 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4073230 con 0x7f2bcc006ea0 2024-03-28T01:06:30.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.193+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc0a76c0 con 0x7f2bcc006ea0 2024-03-28T01:06:30.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.193+0000 7f4afbe00640 1 -- 172.21.15.33:0/1345102249 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f4afc079ca0 con 0x7f4af00039b0 2024-03-28T01:06:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.193+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc0c5590 con 0x7f2bcc006ea0 2024-03-28T01:06:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.192+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c005290 con 0x7ffab81469e0 2024-03-28T01:06:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.194+0000 7fdec8000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec00b2d00 0x7fdec013eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.194+0000 7fdec8000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec006d7c0 con 0x7fdec006d3e0 2024-03-28T01:06:30.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.194+0000 7fdec8000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec00b0aa0 con 0x7fdec013f3e0 2024-03-28T01:06:30.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.194+0000 7fdec8000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00b10f0 con 0x7fdec00b2d00 2024-03-28T01:06:30.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.194+0000 7fddde000640 1 -- 172.21.15.33:0/3163573173 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fddd40419a0 con 0x7fdde0035600 2024-03-28T01:06:30.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.194+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc005390 con 0x7f2bcc006ea0 2024-03-28T01:06:30.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.194+0000 7f7131a00640 1 Processor -- start 2024-03-28T01:06:30.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.195+0000 7fdec4c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec00b2d00 0x7fdec013eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.195+0000 7fdec4c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec00b2d00 0x7fdec013eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48166/0 (socket says 172.21.15.33:48166) 2024-03-28T01:06:30.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.195+0000 7fdec4c00640 1 -- 172.21.15.33:0/879871951 learned_addr learned my addr 172.21.15.33:0/879871951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7fdec4c00640 1 -- 172.21.15.33:0/879871951 >> v1:172.21.15.194:6789/0 conn(0x7fdec013f3e0 legacy=0x7fdec014fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7fdec4c00640 1 -- 172.21.15.33:0/879871951 >> v1:172.21.15.33:6789/0 conn(0x7fdec006d3e0 legacy=0x7fdec00b4a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7fdec4c00640 1 -- 172.21.15.33:0/879871951 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00b0d80 con 0x7fdec00b2d00 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.195+0000 7f7131a00640 1 -- start start 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7f7131a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b0410 0x7f712c120430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7f7131a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f712c0b44a0 0x7f712c1376b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7f7131a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f712c0b4880 0x7f712c136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7f7131a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0b2b90 con 0x7f712c0b44a0 2024-03-28T01:06:30.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7f7131a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0b28b0 con 0x7f712c0b4880 2024-03-28T01:06:30.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7f7131a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0b1840 con 0x7f712c0b0410 2024-03-28T01:06:30.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7ffaaf400640 1 -- 172.21.15.33:0/3517054388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ffaa4024070 con 0x7ffab81469e0 2024-03-28T01:06:30.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7fddde000640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddc806bcd0 0x7fddc806e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.196+0000 7fddde000640 1 -- 172.21.15.33:0/3163573173 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddd40ad6e0 con 0x7fdde0035600 2024-03-28T01:06:30.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.197+0000 7fdde5e00640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddc806bcd0 0x7fddc806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7fd26e600640 1 Processor -- start 2024-03-28T01:06:30.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7fd26e600640 1 -- start start 2024-03-28T01:06:30.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fd26e600640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fd2680b04b0 0x7fd268072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fd26e600640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2680b20e0 con 0x7fd268072990 2024-03-28T01:06:30.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fd26e600640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26806b8c0 con 0x7fd26806b4e0 2024-03-28T01:06:30.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fd26e600640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2680b0d30 con 0x7fd2680b04b0 2024-03-28T01:06:30.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7fdec4c00640 1 --2- 172.21.15.33:0/879871951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec00b2d00 0x7fdec013eea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40098a0 tx=0x7fdeb402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193b8f6bc59c14c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7ffaaf400640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ffa9806bcd0 0x7ffa9806e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7ffaaf400640 1 -- 172.21.15.33:0/3517054388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffaa40923c0 con 0x7ffab81469e0 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.197+0000 7f712aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b0410 0x7f712c120430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.197+0000 7f712b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f712c0b4880 0x7f712c136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7f712aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b0410 0x7f712c120430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48170/0 (socket says 172.21.15.33:48170) 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7f712aa00640 1 -- 172.21.15.33:0/3722242478 learned_addr learned my addr 172.21.15.33:0/3722242478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7f712a000640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f712c0b44a0 0x7f712c1376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7f712aa00640 1 -- 172.21.15.33:0/3722242478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f712c0b4880 msgr2=0x7f712c136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f712aa00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f712c0b4880 0x7f712c136b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f712aa00640 1 -- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f712c0b44a0 msgr2=0x7f712c1376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f712aa00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f712c0b44a0 0x7f712c1376b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f712aa00640 1 -- 172.21.15.33:0/3722242478 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c14af80 con 0x7f712c0b0410 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f712a000640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f712c0b44a0 0x7f712c1376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7f712aa00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b0410 0x7f712c120430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c430 tx=0x7f711c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.198+0000 7fdde5e00640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddc806bcd0 0x7fddc806e190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdde80501b0 tx=0x7fdde806d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.197+0000 7f2bca000640 1 -- 172.21.15.33:0/2259042880 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2bd40733d0 con 0x7f2bcc006ea0 2024-03-28T01:06:30.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f2bca000640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2bb406bbd0 0x7f2bb406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f2bca000640 1 -- 172.21.15.33:0/2259042880 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bd4105830 con 0x7f2bcc006ea0 2024-03-28T01:06:30.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.199+0000 7f2bd1400640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2bb406bbd0 0x7f2bb406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7ffab7400640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ffa9806bcd0 0x7ffa9806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7f2bd1400640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2bb406bbd0 0x7f2bb406e090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc03d9d0 tx=0x7f2bc0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fdebfe00640 1 -- 172.21.15.33:0/879871951 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40317c0 con 0x7fdec00b2d00 2024-03-28T01:06:30.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fdebfe00640 1 -- 172.21.15.33:0/879871951 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb4037070 con 0x7fdec00b2d00 2024-03-28T01:06:30.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fdebfe00640 1 -- 172.21.15.33:0/879871951 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4031b80 con 0x7fdec00b2d00 2024-03-28T01:06:30.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.200+0000 7fd26dc00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd268072990 0x7fd268155e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41116/0 (socket says 172.21.15.33:41116) 2024-03-28T01:06:30.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.201+0000 7fd26dc00640 1 -- 172.21.15.33:0/437572712 learned_addr learned my addr 172.21.15.33:0/437572712 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:30.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.201+0000 7f7122a00640 1 -- 172.21.15.33:0/3722242478 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c011790 con 0x7f712c0b0410 2024-03-28T01:06:30.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7fdec8000640 1 -- 172.21.15.33:0/879871951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec00b2d00 msgr2=0x7fdec013eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7ffab7400640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ffa9806bcd0 0x7ffa9806e190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8000c00 tx=0x7ffaa8009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7fdec8000640 1 --2- 172.21.15.33:0/879871951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec00b2d00 0x7fdec013eea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40098a0 tx=0x7fdeb402b860 comp rx=0 tx=0).stop 2024-03-28T01:06:30.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7fdec8000640 1 -- 172.21.15.33:0/879871951 shutdown_connections 2024-03-28T01:06:30.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7fdec8000640 1 --2- 172.21.15.33:0/879871951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec00b2d00 0x7fdec013eea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7fdec8000640 1 -- 172.21.15.33:0/879871951 >> 172.21.15.33:0/879871951 conn(0x7fdec006b030 msgr2=0x7fdec006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7f7122a00640 1 -- 172.21.15.33:0/3722242478 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c017070 con 0x7f712c0b0410 2024-03-28T01:06:30.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7f7122a00640 1 -- 172.21.15.33:0/3722242478 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c011bb0 con 0x7f712c0b0410 2024-03-28T01:06:30.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c123c00 con 0x7f712c0b0410 2024-03-28T01:06:30.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.202+0000 7fd267e00640 1 --2- 172.21.15.33:0/437572712 >> v2:172.21.15.194:3300/0 conn(0x7fd2680b04b0 0x7fd268072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.203+0000 7fdec8000640 1 -- 172.21.15.33:0/879871951 shutdown_connections 2024-03-28T01:06:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.203+0000 7fdec8000640 1 -- 172.21.15.33:0/879871951 wait complete. 2024-03-28T01:06:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.203+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c1243a0 con 0x7f712c0b0410 2024-03-28T01:06:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.205+0000 7fdec8000640 1 Processor -- start 2024-03-28T01:06:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.205+0000 7fddde000640 1 -- 172.21.15.33:0/3163573173 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fdde00c7d30 con 0x7fdde0035600 2024-03-28T01:06:30.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.204+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c075310 con 0x7f712c0b0410 2024-03-28T01:06:30.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.206+0000 7f7122a00640 1 -- 172.21.15.33:0/3722242478 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f711c01c460 con 0x7f712c0b0410 2024-03-28T01:06:30.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.206+0000 7fdec8000640 1 -- start start 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.206+0000 7fd267e00640 1 -- 172.21.15.33:0/437572712 >> v1:172.21.15.194:6789/0 conn(0x7fd26806b4e0 legacy=0x7fd2680b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.206+0000 7fd267e00640 1 -- 172.21.15.33:0/437572712 >> v1:172.21.15.33:6789/0 conn(0x7fd268072990 legacy=0x7fd268155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.206+0000 7fd267e00640 1 -- 172.21.15.33:0/437572712 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268072e20 con 0x7fd2680b04b0 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec8000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec006d3e0 0x7fdec0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec8000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdec00b2d00 0x7fdec01e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec8000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec013f3e0 0x7fdec01e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec8000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec006bb20 con 0x7fdec013f3e0 2024-03-28T01:06:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec8000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00b5170 con 0x7fdec00b2d00 2024-03-28T01:06:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.207+0000 7f7122a00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f710c06bbd0 0x7f710c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.207+0000 7f7122a00640 1 -- 172.21.15.33:0/3722242478 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f711c08a8d0 con 0x7f712c0b0410 2024-03-28T01:06:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec8000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec009c4d0 con 0x7fdec006d3e0 2024-03-28T01:06:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fd267e00640 1 --2- 172.21.15.33:0/437572712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2680b04b0 0x7fd268072450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd2540097d0 tx=0x7fd25402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c0a4766bd88097 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7f712a000640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f710c06bbd0 0x7f710c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec6000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec013f3e0 0x7fdec01e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec6000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec013f3e0 0x7fdec01e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36514/0 (socket says 172.21.15.33:36514) 2024-03-28T01:06:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.208+0000 7fdec6000640 1 -- 172.21.15.33:0/483784233 learned_addr learned my addr 172.21.15.33:0/483784233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.209+0000 7f712a000640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f710c06bbd0 0x7f710c06e090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f712c13a970 tx=0x7f7118008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.209+0000 7fdec6000640 1 -- 172.21.15.33:0/483784233 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec006d3e0 msgr2=0x7fdec0071940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T01:06:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.209+0000 7fdec6000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec006d3e0 0x7fdec0071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.209+0000 7fdec6000640 1 -- 172.21.15.33:0/483784233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdec00b2d00 msgr2=0x7fdec01e4650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T01:06:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.209+0000 7fdec6000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdec00b2d00 0x7fdec01e4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.209+0000 7fdec6000640 1 -- 172.21.15.33:0/483784233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec009ac60 con 0x7fdec013f3e0 2024-03-28T01:06:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.210+0000 7fdec6000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec013f3e0 0x7fdec01e4b90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800b990 tx=0x7fdeb800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.213+0000 7fdebd600640 1 -- 172.21.15.33:0/483784233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8019070 con 0x7fdec013f3e0 2024-03-28T01:06:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.213+0000 7fdebd600640 1 -- 172.21.15.33:0/483784233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb800cba0 con 0x7fdec013f3e0 2024-03-28T01:06:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.213+0000 7fdebd600640 1 -- 172.21.15.33:0/483784233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8014440 con 0x7fdec013f3e0 2024-03-28T01:06:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.214+0000 7ffaaf400640 1 -- 172.21.15.33:0/3517054388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ffaa4019070 con 0x7ffab81469e0 2024-03-28T01:06:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.214+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec009af40 con 0x7fdec013f3e0 2024-03-28T01:06:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.214+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec009c1f0 con 0x7fdec013f3e0 2024-03-28T01:06:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.216+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec009b400 con 0x7fdec013f3e0 2024-03-28T01:06:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.219+0000 7f2bca000640 1 -- 172.21.15.33:0/2259042880 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2bd40a2080 con 0x7f2bcc006ea0 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.219+0000 7fd267400640 1 -- 172.21.15.33:0/437572712 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254035020 con 0x7fd2680b04b0 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.219+0000 7fd267400640 1 -- 172.21.15.33:0/437572712 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd254031810 con 0x7fd2680b04b0 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.219+0000 7fd267400640 1 -- 172.21.15.33:0/437572712 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254031be0 con 0x7fd2680b04b0 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.220+0000 7fd26e600640 1 -- 172.21.15.33:0/437572712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2680b04b0 msgr2=0x7fd268072450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.220+0000 7fd26e600640 1 --2- 172.21.15.33:0/437572712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2680b04b0 0x7fd268072450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd2540097d0 tx=0x7fd25402b8b0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.220+0000 7fd26e600640 1 -- 172.21.15.33:0/437572712 shutdown_connections 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.220+0000 7fd26e600640 1 --2- 172.21.15.33:0/437572712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2680b04b0 0x7fd268072450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.220+0000 7fd26e600640 1 -- 172.21.15.33:0/437572712 >> 172.21.15.33:0/437572712 conn(0x7fd2680b89c0 msgr2=0x7fd2680b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.221+0000 7fd26e600640 1 -- 172.21.15.33:0/437572712 shutdown_connections 2024-03-28T01:06:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.221+0000 7fd26e600640 1 -- 172.21.15.33:0/437572712 wait complete. 2024-03-28T01:06:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.223+0000 7fd26e600640 1 Processor -- start 2024-03-28T01:06:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.224+0000 7fd26e600640 1 -- start start 2024-03-28T01:06:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.218+0000 7fdebd600640 1 -- 172.21.15.33:0/483784233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdeb801c070 con 0x7fdec013f3e0 2024-03-28T01:06:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.220+0000 7fdebd600640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdea006bbd0 0x7fdea006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.224+0000 7fdebd600640 1 -- 172.21.15.33:0/483784233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdeb8056c70 con 0x7fdec013f3e0 2024-03-28T01:06:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.226+0000 7fd26e600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd26806b4e0 0x7fd2681d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.226+0000 7fd26e600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd268072990 0x7fd2681e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.227+0000 7fd26e600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2680b04b0 0x7fd2681e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7fd26e600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680b4600 con 0x7fd26806b4e0 2024-03-28T01:06:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.225+0000 7fdec5600640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdea006bbd0 0x7fdea006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7fd26e600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680b09c0 con 0x7fd268072990 2024-03-28T01:06:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7fd267e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd268072990 0x7fd2681e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7fd267e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd268072990 0x7fd2681e9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44520/0 (socket says 172.21.15.33:44520) 2024-03-28T01:06:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7fd267e00640 1 -- 172.21.15.33:0/2601988456 learned_addr learned my addr 172.21.15.33:0/2601988456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7fd26d200640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd26806b4e0 0x7fd2681d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7fd26dc00640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2680b04b0 0x7fd2681e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.228+0000 7f7122a00640 1 -- 172.21.15.33:0/3722242478 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f711c0593b0 con 0x7f712c0b0410 2024-03-28T01:06:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.229+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680b4780 con 0x7fd2680b04b0 2024-03-28T01:06:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.229+0000 7fdec5600640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdea006bbd0 0x7fdea006e090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009670 tx=0x7fdeac006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.231+0000 7fd26d200640 1 -- 172.21.15.33:0/2601988456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2680b04b0 msgr2=0x7fd2681e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.231+0000 7fd26d200640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2680b04b0 0x7fd2681e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.231+0000 7fd26d200640 1 -- 172.21.15.33:0/2601988456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd268072990 msgr2=0x7fd2681e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.231+0000 7fd26d200640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd268072990 0x7fd2681e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.231+0000 7fd26d200640 1 -- 172.21.15.33:0/2601988456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2681d6820 con 0x7fd26806b4e0 2024-03-28T01:06:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.232+0000 7fd26d200640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd26806b4e0 0x7fd2681d55c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00c390 tx=0x7fd25c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.234+0000 7fd264c00640 1 -- 172.21.15.33:0/2601988456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c017070 con 0x7fd26806b4e0 2024-03-28T01:06:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.235+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2681ed7f0 con 0x7fd26806b4e0 2024-03-28T01:06:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.235+0000 7fd264c00640 1 -- 172.21.15.33:0/2601988456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c0052d0 con 0x7fd26806b4e0 2024-03-28T01:06:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.235+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2681eddb0 con 0x7fd26806b4e0 2024-03-28T01:06:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.236+0000 7fd264c00640 1 -- 172.21.15.33:0/2601988456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c003830 con 0x7fd26806b4e0 2024-03-28T01:06:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.236+0000 7fdebd600640 1 -- 172.21.15.33:0/483784233 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fdeb80592d0 con 0x7fdec013f3e0 2024-03-28T01:06:30.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.240+0000 7fd264c00640 1 -- 172.21.15.33:0/2601988456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd25c01a070 con 0x7fd26806b4e0 2024-03-28T01:06:30.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.241+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd268075310 con 0x7fd26806b4e0 2024-03-28T01:06:30.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.251+0000 7fd264c00640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fd24806bc00 0x7fd24806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.252+0000 7fd264c00640 1 -- 172.21.15.33:0/2601988456 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd25c089d00 con 0x7fd26806b4e0 2024-03-28T01:06:30.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.252+0000 7fd264c00640 1 -- 172.21.15.33:0/2601988456 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fd25c058190 con 0x7fd26806b4e0 2024-03-28T01:06:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.255+0000 7fd267e00640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fd24806bc00 0x7fd24806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.260+0000 7fd267e00640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fd24806bc00 0x7fd24806e0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd25403c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.417+0000 7fdbb7000640 1 Processor -- start 2024-03-28T01:06:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.418+0000 7fdbb7000640 1 -- start start 2024-03-28T01:06:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.420+0000 7fdbb7000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fdbb00b51d0 0x7fdbb00b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.420+0000 7fdbb7000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb00b55b0 con 0x7fdbb00b8d00 2024-03-28T01:06:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.420+0000 7fdbb7000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb00b9190 con 0x7fdbb00b42f0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.420+0000 7fdbb7000640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb014a820 con 0x7fdbb00b51d0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.420+0000 7fdbb5c00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fdbb00b51d0 0x7fdbb00b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.420+0000 7fdbb6600640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7fdbb00b42f0 0x7fdbb0151b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37932/0 (socket says 172.21.15.33:37932) 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.420+0000 7fdbb6600640 1 -- 172.21.15.33:0/4191121598 learned_addr learned my addr 172.21.15.33:0/4191121598 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.424+0000 7fdbafe00640 1 -- 172.21.15.33:0/4191121598 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757886222 0 0) 0x7fdbb00b9190 con 0x7fdbb00b42f0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.424+0000 7fdbafe00640 1 -- 172.21.15.33:0/4191121598 --> v1:172.21.15.194:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb98003520 con 0x7fdbb00b42f0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.424+0000 7fdbb5c00640 1 -- 172.21.15.33:0/4191121598 >> v1:172.21.15.194:6789/0 conn(0x7fdbb00b42f0 legacy=0x7fdbb0151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.424+0000 7fdbafe00640 1 -- 172.21.15.33:0/4191121598 <== mon.1 v1:172.21.15.194:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 316044512 0 0) 0x7fdb98003520 con 0x7fdbb00b42f0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.424+0000 7fdbb5c00640 1 -- 172.21.15.33:0/4191121598 >> v1:172.21.15.33:6789/0 conn(0x7fdbb00b8d00 legacy=0x7fdbb014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.424+0000 7fdbb5c00640 1 -- 172.21.15.33:0/4191121598 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb00b9800 con 0x7fdbb00b51d0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.433+0000 7fdbb5c00640 1 --2- 172.21.15.33:0/4191121598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 0x7fdbb00b3db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009d90 tx=0x7fdba802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40beef2b6b956c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.434+0000 7fdbafe00640 1 -- 172.21.15.33:0/4191121598 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8039070 con 0x7fdbb00b51d0 2024-03-28T01:06:30.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.435+0000 7fdbafe00640 1 -- 172.21.15.33:0/4191121598 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba802c8b0 con 0x7fdbb00b51d0 2024-03-28T01:06:30.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.435+0000 7fdbafe00640 1 -- 172.21.15.33:0/4191121598 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c001670 con 0x7fdbb00b51d0 2024-03-28T01:06:30.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.438+0000 7fdbb7000640 1 -- 172.21.15.33:0/4191121598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 msgr2=0x7fdbb00b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.438+0000 7fdbb7000640 1 --2- 172.21.15.33:0/4191121598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 0x7fdbb00b3db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009d90 tx=0x7fdba802bd80 comp rx=0 tx=0).stop 2024-03-28T01:06:30.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.446+0000 7fdbb7000640 1 -- 172.21.15.33:0/4191121598 shutdown_connections 2024-03-28T01:06:30.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.446+0000 7fdbb7000640 1 --2- 172.21.15.33:0/4191121598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 0x7fdbb00b3db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.446+0000 7fdbb7000640 1 -- 172.21.15.33:0/4191121598 >> 172.21.15.33:0/4191121598 conn(0x7fdbb0072940 msgr2=0x7fdbb0072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.447+0000 7fdbb7000640 1 -- 172.21.15.33:0/4191121598 shutdown_connections 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.448+0000 7fdbb7000640 1 -- 172.21.15.33:0/4191121598 wait complete. 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.450+0000 7fdbb7000640 1 Processor -- start 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.464+0000 7fdbb7000640 1 -- start start 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.465+0000 7fdbb7000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb00b42f0 0x7fdbb01d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.465+0000 7fdbb7000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 0x7fdbb00bca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.465+0000 7fdbb7000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb01d03c0 0x7fdbb01d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.465+0000 7fdbb7000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00b6b80 con 0x7fdbb01d03c0 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.465+0000 7fdbb7000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00b39c0 con 0x7fdbb00b51d0 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.465+0000 7fdbb7000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00b6d00 con 0x7fdbb00b42f0 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.468+0000 7fdbb6600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb01d03c0 0x7fdbb01d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.468+0000 7fdbb6600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb01d03c0 0x7fdbb01d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36536/0 (socket says 172.21.15.33:36536) 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.468+0000 7fdbb6600640 1 -- 172.21.15.33:0/2991297070 learned_addr learned my addr 172.21.15.33:0/2991297070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.473+0000 7fdbb6600640 1 -- 172.21.15.33:0/2991297070 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb00b42f0 msgr2=0x7fdbb01d1e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T01:06:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.473+0000 7fdbb6600640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb00b42f0 0x7fdbb01d1e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.473+0000 7fdbb6600640 1 -- 172.21.15.33:0/2991297070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 msgr2=0x7fdbb00bca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T01:06:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.473+0000 7fdbb6600640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 0x7fdbb00bca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.473+0000 7fdbb6600640 1 -- 172.21.15.33:0/2991297070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb01d16b0 con 0x7fdbb01d03c0 2024-03-28T01:06:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.473+0000 7fdbb6600640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb01d03c0 0x7fdbb01d0d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00c7b0 tx=0x7fdb9c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.476+0000 7fdbad600640 1 -- 172.21.15.33:0/2991297070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017070 con 0x7fdbb01d03c0 2024-03-28T01:06:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.476+0000 7fdbad600640 1 -- 172.21.15.33:0/2991297070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c005ad0 con 0x7fdbb01d03c0 2024-03-28T01:06:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.476+0000 7fdbad600640 1 -- 172.21.15.33:0/2991297070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017070 con 0x7fdbb01d03c0 2024-03-28T01:06:30.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.478+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb0144710 con 0x7fdbb01d03c0 2024-03-28T01:06:30.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.485+0000 7fdbad600640 1 -- 172.21.15.33:0/2991297070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdb9c01a070 con 0x7fdbb01d03c0 2024-03-28T01:06:30.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.486+0000 7fdbad600640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdb9006bdd0 0x7fdb9006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:30.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.496+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb0143680 con 0x7fdbb01d03c0 2024-03-28T01:06:30.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.502+0000 7fdbb5c00640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdb9006bdd0 0x7fdb9006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.502+0000 7fdbad600640 1 -- 172.21.15.33:0/2991297070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb9c055e10 con 0x7fdbb01d03c0 2024-03-28T01:06:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.510+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb0075310 con 0x7fdbb01d03c0 2024-03-28T01:06:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.513+0000 7fdbb5c00640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdb9006bdd0 0x7fdb9006e290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdba8036000 tx=0x7fdba8030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:30.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.529+0000 7fdbad600640 1 -- 172.21.15.33:0/2991297070 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fdb9c058470 con 0x7fdbb01d03c0 2024-03-28T01:06:30.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.643+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4b0003c820 con 0x7f4af00039b0 2024-03-28T01:06:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.644+0000 7f4afbe00640 1 -- 172.21.15.33:0/1345102249 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4afc07dd40 con 0x7f4af00039b0 2024-03-28T01:06:30.647 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-03-28T01:06:30.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.648+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f4ae806be00 msgr2=0x7f4ae806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.648+0000 7f4b0f200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f4ae806be00 0x7f4ae806e2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b000ced20 tx=0x7f4af4008040 comp rx=0 tx=0).stop 2024-03-28T01:06:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.648+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af00039b0 msgr2=0x7f4b000c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.648+0000 7f4b0f200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af00039b0 0x7f4b000c2460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4afc02bf10 tx=0x7f4afc001230 comp rx=0 tx=0).stop 2024-03-28T01:06:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 shutdown_connections 2024-03-28T01:06:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f4ae806be00 0x7f4ae806e2c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b001614d0 0x7f4b000c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b000ce940 0x7f4b000c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 --2- 172.21.15.33:0/1345102249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4af00039b0 0x7f4b000c2460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 >> 172.21.15.33:0/1345102249 conn(0x7f4b0003ec80 msgr2=0x7f4b000401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 shutdown_connections 2024-03-28T01:06:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.649+0000 7f4b0f200640 1 -- 172.21.15.33:0/1345102249 wait complete. 2024-03-28T01:06:30.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.650+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdde0005390 con 0x7fdde0035600 2024-03-28T01:06:30.655 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2024-03-28T01:06:30.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.652+0000 7fddde000640 1 -- 172.21.15.33:0/3163573173 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fddd404f020 con 0x7fdde0035600 2024-03-28T01:06:30.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.667+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddc806bcd0 msgr2=0x7fddc806e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.667+0000 7fdde7200640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddc806bcd0 0x7fddc806e190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdde80501b0 tx=0x7fdde806d040 comp rx=0 tx=0).stop 2024-03-28T01:06:30.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.667+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0035600 msgr2=0x7fdde00c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.667+0000 7fdde7200640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0035600 0x7fdde00c9780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fddd402e550 tx=0x7fddd40046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.668+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffa8c005580 con 0x7ffab81469e0 2024-03-28T01:06:30.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.669+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 shutdown_connections 2024-03-28T01:06:30.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.669+0000 7fdde7200640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddc806bcd0 0x7fddc806e190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.669+0000 7fdde7200640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdde00359e0 0x7fdde00329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.669+0000 7fdde7200640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0035600 0x7fdde00c9780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.669+0000 7fdde7200640 1 --2- 172.21.15.33:0/3163573173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdde0033d10 0x7fdde00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.669+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 >> 172.21.15.33:0/3163573173 conn(0x7fdde003a8a0 msgr2=0x7fdde003b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.670+0000 7ffaaf400640 1 -- 172.21.15.33:0/3517054388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffaa40608d0 con 0x7ffab81469e0 2024-03-28T01:06:30.673 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:30.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.672+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 shutdown_connections 2024-03-28T01:06:30.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.672+0000 7fdde7200640 1 -- 172.21.15.33:0/3163573173 wait complete. 2024-03-28T01:06:30.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.672+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f712c14b260 con 0x7f712c0b0410 2024-03-28T01:06:30.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.673+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2bcc037d30 con 0x7f2bcc006ea0 2024-03-28T01:06:30.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.673+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ffa9806bcd0 msgr2=0x7ffa9806e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.673+0000 7ffaacc00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ffa9806bcd0 0x7ffa9806e190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8000c00 tx=0x7ffaa8009030 comp rx=0 tx=0).stop 2024-03-28T01:06:30.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.673+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffab81469e0 msgr2=0x7ffab81dcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.673+0000 7ffaacc00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffab81469e0 0x7ffab81dcb10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4001b00 tx=0x7ffaa40164a0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7f2bca000640 1 -- 172.21.15.33:0/2259042880 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bd40d8e20 con 0x7f2bcc006ea0 2024-03-28T01:06:30.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 shutdown_connections 2024-03-28T01:06:30.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7ffaacc00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ffa9806bcd0 0x7ffa9806e190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7ffaacc00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffab8146e20 0x7ffab81dd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7ffaacc00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffab81469e0 0x7ffab81dcb10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7ffaacc00640 1 --2- 172.21.15.33:0/3517054388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffab8149f00 0x7ffab81458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 >> 172.21.15.33:0/3517054388 conn(0x7ffab80b69d0 msgr2=0x7ffab80b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.674+0000 7f7122a00640 1 -- 172.21.15.33:0/3722242478 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f711c058cb0 con 0x7f712c0b0410 2024-03-28T01:06:30.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.675+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 shutdown_connections 2024-03-28T01:06:30.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.675+0000 7ffaacc00640 1 -- 172.21.15.33:0/3517054388 wait complete. 2024-03-28T01:06:30.678 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-03-28T01:06:30.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-03-28T01:06:30.679 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-03-28T01:06:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.678+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f710c06bbd0 msgr2=0x7f710c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.678+0000 7f7131a00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f710c06bbd0 0x7f710c06e090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f712c13a970 tx=0x7f7118008040 comp rx=0 tx=0).stop 2024-03-28T01:06:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.678+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b0410 msgr2=0x7f712c120430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.678+0000 7f7131a00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b0410 0x7f712c120430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c430 tx=0x7f711c00c8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2bb406bbd0 msgr2=0x7f2bb406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2bb406bbd0 0x7f2bb406e090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc03d9d0 tx=0x7f2bc0008040 comp rx=0 tx=0).stop 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 shutdown_connections 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f7131a00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f710c06bbd0 0x7f710c06e090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f7131a00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f712c0b4880 0x7f712c136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f7131a00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f712c0b44a0 0x7f712c1376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f7131a00640 1 --2- 172.21.15.33:0/3722242478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f712c0b0410 0x7f712c120430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 >> 172.21.15.33:0/3722242478 conn(0x7f712c0b5740 msgr2=0x7f712c0b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bcc006ea0 msgr2=0x7f2bcc0a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bcc006ea0 0x7f2bcc0a6d20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4069ea0 tx=0x7f2bd4068b90 comp rx=0 tx=0).stop 2024-03-28T01:06:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 shutdown_connections 2024-03-28T01:06:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2bb406bbd0 0x7f2bb406e090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bcc0365a0 0x7f2bcc0a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2bcc035b40 0x7f2bcc0c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 --2- 172.21.15.33:0/2259042880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bcc006ea0 0x7f2bcc0a6d20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.679+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 >> 172.21.15.33:0/2259042880 conn(0x7f2bcc03a8a0 msgr2=0x7f2bcc0064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.680+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 shutdown_connections 2024-03-28T01:06:30.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.680+0000 7f7131a00640 1 -- 172.21.15.33:0/3722242478 wait complete. 2024-03-28T01:06:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.680+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 shutdown_connections 2024-03-28T01:06:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.680+0000 7f2bd3200640 1 -- 172.21.15.33:0/2259042880 wait complete. 2024-03-28T01:06:30.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-03-28T01:06:30.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-03-28T01:06:30.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-03-28T01:06:30.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-03-28T01:06:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.721+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdec0075380 con 0x7fdec013f3e0 2024-03-28T01:06:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.721+0000 7fdebd600640 1 -- 172.21.15.33:0/483784233 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdeb80592d0 con 0x7fdec013f3e0 2024-03-28T01:06:30.724 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdea006bbd0 msgr2=0x7fdea006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdea006bbd0 0x7fdea006e090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009670 tx=0x7fdeac006bd0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec013f3e0 msgr2=0x7fdec01e4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec013f3e0 0x7fdec01e4b90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800b990 tx=0x7fdeb800be50 comp rx=0 tx=0).stop 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 shutdown_connections 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdea006bbd0 0x7fdea006e090 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec013f3e0 0x7fdec01e4b90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdec00b2d00 0x7fdec01e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 --2- 172.21.15.33:0/483784233 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdec006d3e0 0x7fdec0071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.724+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 >> 172.21.15.33:0/483784233 conn(0x7fdec006b030 msgr2=0x7fdec00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.725+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 shutdown_connections 2024-03-28T01:06:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.725+0000 7fdec8000640 1 -- 172.21.15.33:0/483784233 wait complete. 2024-03-28T01:06:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.747+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd2681edf80 con 0x7fd26806b4e0 2024-03-28T01:06:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.747+0000 7fd264c00640 1 -- 172.21.15.33:0/2601988456 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd25c05c230 con 0x7fd26806b4e0 2024-03-28T01:06:30.750 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-03-28T01:06:30.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fd24806bc00 msgr2=0x7fd24806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fd24806bc00 0x7fd24806e0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd25403c040 comp rx=0 tx=0).stop 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd26806b4e0 msgr2=0x7fd2681d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd26806b4e0 0x7fd2681d55c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00c390 tx=0x7fd25c00c850 comp rx=0 tx=0).stop 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 shutdown_connections 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fd24806bc00 0x7fd24806e0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2680b04b0 0x7fd2681e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd268072990 0x7fd2681e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 --2- 172.21.15.33:0/2601988456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd26806b4e0 0x7fd2681d55c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.750+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 >> 172.21.15.33:0/2601988456 conn(0x7fd2680b89c0 msgr2=0x7fd2680b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.751+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 shutdown_connections 2024-03-28T01:06:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.751+0000 7fd26e600640 1 -- 172.21.15.33:0/2601988456 wait complete. 2024-03-28T01:06:30.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-03-28T01:06:30.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.880+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdbb0144ee0 con 0x7fdbb01d03c0 2024-03-28T01:06:30.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.881+0000 7fdbad600640 1 -- 172.21.15.33:0/2991297070 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb9c057d70 con 0x7fdbb01d03c0 2024-03-28T01:06:30.884 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2024-03-28T01:06:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdb9006bdd0 msgr2=0x7fdb9006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdb9006bdd0 0x7fdb9006e290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdba8036000 tx=0x7fdba8030000 comp rx=0 tx=0).stop 2024-03-28T01:06:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb01d03c0 msgr2=0x7fdbb01d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb01d03c0 0x7fdbb01d0d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00c7b0 tx=0x7fdb9c00cc70 comp rx=0 tx=0).stop 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 shutdown_connections 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdb9006bdd0 0x7fdb9006e290 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb01d03c0 0x7fdbb01d0d00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb00b51d0 0x7fdbb00bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.883+0000 7fdbb7000640 1 --2- 172.21.15.33:0/2991297070 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb00b42f0 0x7fdbb01d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.884+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 >> 172.21.15.33:0/2991297070 conn(0x7fdbb0072940 msgr2=0x7fdbb00b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.884+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 shutdown_connections 2024-03-28T01:06:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:30.884+0000 7fdbb7000640 1 -- 172.21.15.33:0/2991297070 wait complete. 2024-03-28T01:06:30.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-03-28T01:06:30.903 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:31.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-28T01:06:31.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T01:06:31.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T01:06:31.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T01:06:31.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-28T01:06:31.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T01:06:31.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-28T01:06:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.964+0000 7fb16e200640 1 Processor -- start 2024-03-28T01:06:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.964+0000 7fb16e200640 1 -- start start 2024-03-28T01:06:31.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.966+0000 7fb16e200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fb168146e00 0x7fb1680af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:31.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.967+0000 7fb16e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168147620 0x7fb16806c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.967+0000 7fb16e200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16806de10 con 0x7fb1681471e0 2024-03-28T01:06:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.967+0000 7fb16e200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1680affd0 con 0x7fb168146e00 2024-03-28T01:06:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.967+0000 7fb16e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680b02b0 con 0x7fb168147620 2024-03-28T01:06:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.968+0000 7fb166a00640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7fb1681471e0 0x7fb16806c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37934/0 (socket says 172.21.15.33:37934) 2024-03-28T01:06:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.968+0000 7fb166a00640 1 -- 172.21.15.33:0/254305425 learned_addr learned my addr 172.21.15.33:0/254305425 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.968+0000 7fb167400640 1 --2- 172.21.15.33:0/254305425 >> v2:172.21.15.194:3300/0 conn(0x7fb168146e00 0x7fb1680af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.968+0000 7fb167e00640 1 --2- 172.21.15.33:0/254305425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168147620 0x7fb16806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.969+0000 7fb167e00640 1 -- 172.21.15.33:0/254305425 >> v2:172.21.15.194:3300/0 conn(0x7fb168146e00 msgr2=0x7fb1680af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.969+0000 7fb167e00640 1 --2- 172.21.15.33:0/254305425 >> v2:172.21.15.194:3300/0 conn(0x7fb168146e00 0x7fb1680af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.969+0000 7fb166000640 1 -- 172.21.15.33:0/254305425 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797447253 0 0) 0x7fb16806de10 con 0x7fb1681471e0 2024-03-28T01:06:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.969+0000 7fb167e00640 1 -- 172.21.15.33:0/254305425 >> v1:172.21.15.194:6789/0 conn(0x7fb1681471e0 legacy=0x7fb16806c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.969+0000 7fb167e00640 1 -- 172.21.15.33:0/254305425 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680ae340 con 0x7fb168147620 2024-03-28T01:06:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.971+0000 7fb167e00640 1 --2- 172.21.15.33:0/254305425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168147620 0x7fb16806c870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb1540097a0 tx=0x7fb15402dbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765a35475f58bed0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.972+0000 7fb166000640 1 -- 172.21.15.33:0/254305425 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15403b070 con 0x7fb168147620 2024-03-28T01:06:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.973+0000 7fb166000640 1 -- 172.21.15.33:0/254305425 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15402e970 con 0x7fb168147620 2024-03-28T01:06:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.973+0000 7fb166000640 1 -- 172.21.15.33:0/254305425 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15402ed40 con 0x7fb168147620 2024-03-28T01:06:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.974+0000 7fb16e200640 1 -- 172.21.15.33:0/254305425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168147620 msgr2=0x7fb16806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.974+0000 7fb16e200640 1 --2- 172.21.15.33:0/254305425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168147620 0x7fb16806c870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb1540097a0 tx=0x7fb15402dbc0 comp rx=0 tx=0).stop 2024-03-28T01:06:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.975+0000 7fb16e200640 1 -- 172.21.15.33:0/254305425 shutdown_connections 2024-03-28T01:06:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.975+0000 7fb16e200640 1 --2- 172.21.15.33:0/254305425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168147620 0x7fb16806c870 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.975+0000 7fb16e200640 1 --2- 172.21.15.33:0/254305425 >> v2:172.21.15.194:3300/0 conn(0x7fb168146e00 0x7fb1680af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.975+0000 7fb16e200640 1 -- 172.21.15.33:0/254305425 >> 172.21.15.33:0/254305425 conn(0x7fb1680b69d0 msgr2=0x7fb1680b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.976+0000 7fb16e200640 1 -- 172.21.15.33:0/254305425 shutdown_connections 2024-03-28T01:06:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.976+0000 7fb16e200640 1 -- 172.21.15.33:0/254305425 wait complete. 2024-03-28T01:06:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.979+0000 7fb16e200640 1 Processor -- start 2024-03-28T01:06:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.980+0000 7fb16e200640 1 -- start start 2024-03-28T01:06:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.981+0000 7fb16e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168146e00 0x7fb1680b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.981+0000 7fb16e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1681471e0 0x7fb16806f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.981+0000 7fb16e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb168147620 0x7fb16806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.981+0000 7fb16e200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680adfd0 con 0x7fb1681471e0 2024-03-28T01:06:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.981+0000 7fb16e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680ade50 con 0x7fb168147620 2024-03-28T01:06:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.981+0000 7fb16e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16806cdb0 con 0x7fb168146e00 2024-03-28T01:06:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.983+0000 7fb166a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1681471e0 0x7fb16806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.983+0000 7fb167e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb168147620 0x7fb16806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.983+0000 7fb167e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb168147620 0x7fb16806fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44540/0 (socket says 172.21.15.33:44540) 2024-03-28T01:06:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.983+0000 7fb167e00640 1 -- 172.21.15.33:0/4168850860 learned_addr learned my addr 172.21.15.33:0/4168850860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.984+0000 7fb167400640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168146e00 0x7fb1680b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.985+0000 7fb167e00640 1 -- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168146e00 msgr2=0x7fb1680b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.985+0000 7fb167e00640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168146e00 0x7fb1680b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.985+0000 7fb167e00640 1 -- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1681471e0 msgr2=0x7fb16806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.985+0000 7fb167e00640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1681471e0 0x7fb16806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.985+0000 7fb167e00640 1 -- 172.21.15.33:0/4168850860 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1681d0ea0 con 0x7fb168147620 2024-03-28T01:06:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.985+0000 7fb166a00640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1681471e0 0x7fb16806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.986+0000 7fb167e00640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb168147620 0x7fb16806fbc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb154039040 tx=0x7fb1540046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.986+0000 7fb15f400640 1 -- 172.21.15.33:0/4168850860 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154046070 con 0x7fb168147620 2024-03-28T01:06:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.986+0000 7fb15f400640 1 -- 172.21.15.33:0/4168850860 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb154041460 con 0x7fb168147620 2024-03-28T01:06:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.986+0000 7fb15f400640 1 -- 172.21.15.33:0/4168850860 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154046070 con 0x7fb168147620 2024-03-28T01:06:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.986+0000 7fb167400640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168146e00 0x7fb1680b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.988+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1681448b0 con 0x7fb168147620 2024-03-28T01:06:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.988+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1681d13f0 con 0x7fb168147620 2024-03-28T01:06:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.990+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb168075380 con 0x7fb168147620 2024-03-28T01:06:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.992+0000 7fb15f400640 1 -- 172.21.15.33:0/4168850860 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb15403f070 con 0x7fb168147620 2024-03-28T01:06:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.993+0000 7fb15f400640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb14806bdd0 0x7fb14806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.994+0000 7fb15f400640 1 -- 172.21.15.33:0/4168850860 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1540783e0 con 0x7fb168147620 2024-03-28T01:06:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.994+0000 7fb167400640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb14806bdd0 0x7fb14806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:31.995+0000 7fb167400640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb14806bdd0 0x7fb14806e290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb1680b4e80 tx=0x7fb1600074c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.008+0000 7f1db5e00640 1 Processor -- start 2024-03-28T01:06:32.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.009+0000 7f1db5e00640 1 -- start start 2024-03-28T01:06:32.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.010+0000 7f1db5e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1db0066dd0 0x7f1db0072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.010+0000 7f1db5e00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00b5a70 con 0x7f1db00b5690 2024-03-28T01:06:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.010+0000 7f1db5e00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00af9d0 con 0x7f1db0073320 2024-03-28T01:06:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.010+0000 7f1db5e00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00af7a0 con 0x7f1db0066dd0 2024-03-28T01:06:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.011+0000 7f1daea00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1db00b5690 0x7f1db0143680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41130/0 (socket says 172.21.15.33:41130) 2024-03-28T01:06:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.011+0000 7f1daea00640 1 -- 172.21.15.33:0/4116994417 learned_addr learned my addr 172.21.15.33:0/4116994417 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.011+0000 7fb15f400640 1 -- 172.21.15.33:0/4168850860 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb15407aa40 con 0x7fb168147620 2024-03-28T01:06:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.011+0000 7f1dae000640 1 --2- 172.21.15.33:0/4116994417 >> v2:172.21.15.33:3300/0 conn(0x7f1db0066dd0 0x7f1db0072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.012+0000 7f1dae000640 1 -- 172.21.15.33:0/4116994417 >> v1:172.21.15.194:6789/0 conn(0x7f1db0073320 legacy=0x7f1db013d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.012+0000 7f1dae000640 1 -- 172.21.15.33:0/4116994417 >> v1:172.21.15.33:6789/0 conn(0x7f1db00b5690 legacy=0x7f1db0143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.012+0000 7f1dae000640 1 -- 172.21.15.33:0/4116994417 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00afcb0 con 0x7f1db0066dd0 2024-03-28T01:06:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.014+0000 7f1dae000640 1 --2- 172.21.15.33:0/4116994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db0066dd0 0x7f1db0072de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1da00097a0 tx=0x7f1da002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0dcda4963a5728 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.014+0000 7f1dad600640 1 -- 172.21.15.33:0/4116994417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0037070 con 0x7f1db0066dd0 2024-03-28T01:06:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.015+0000 7f1dad600640 1 -- 172.21.15.33:0/4116994417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0030710 con 0x7f1db0066dd0 2024-03-28T01:06:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.015+0000 7f1dad600640 1 -- 172.21.15.33:0/4116994417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0030ae0 con 0x7f1db0066dd0 2024-03-28T01:06:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.015+0000 7f1db5e00640 1 -- 172.21.15.33:0/4116994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db0066dd0 msgr2=0x7f1db0072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.015+0000 7f1db5e00640 1 --2- 172.21.15.33:0/4116994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db0066dd0 0x7f1db0072de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1da00097a0 tx=0x7f1da002b710 comp rx=0 tx=0).stop 2024-03-28T01:06:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.016+0000 7f1db5e00640 1 -- 172.21.15.33:0/4116994417 shutdown_connections 2024-03-28T01:06:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.016+0000 7f1db5e00640 1 --2- 172.21.15.33:0/4116994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db0066dd0 0x7f1db0072de0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.016+0000 7f1db5e00640 1 -- 172.21.15.33:0/4116994417 >> 172.21.15.33:0/4116994417 conn(0x7f1db00b9800 msgr2=0x7f1db00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.017+0000 7f1db5e00640 1 -- 172.21.15.33:0/4116994417 shutdown_connections 2024-03-28T01:06:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.018+0000 7f1db5e00640 1 -- 172.21.15.33:0/4116994417 wait complete. 2024-03-28T01:06:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.020+0000 7f1db5e00640 1 Processor -- start 2024-03-28T01:06:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.020+0000 7f1db5e00640 1 -- start start 2024-03-28T01:06:32.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.021+0000 7f3405200640 1 Processor -- start 2024-03-28T01:06:32.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.021+0000 7f1db5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db0066dd0 0x7f1db009dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.021+0000 7f1db5e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db00b5690 0x7f1db00b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1db5e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1db01d9230 0x7f1db01d96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1db5e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b0350 con 0x7f1db00b5690 2024-03-28T01:06:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1dae000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db00b5690 0x7f1db00b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1db5e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b01d0 con 0x7f1db0066dd0 2024-03-28T01:06:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1dae000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db00b5690 0x7f1db00b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36566/0 (socket says 172.21.15.33:36566) 2024-03-28T01:06:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1dae000640 1 -- 172.21.15.33:0/3984323615 learned_addr learned my addr 172.21.15.33:0/3984323615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1daea00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db0066dd0 0x7f1db009dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db0067260 con 0x7f1db01d9230 2024-03-28T01:06:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.022+0000 7f1daf400640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1db01d9230 0x7f1db01d96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.021+0000 7f3405200640 1 -- start start 2024-03-28T01:06:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.023+0000 7f3405200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340006d3e0 0x7f34000af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.023+0000 7f3405200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340006d7c0 con 0x7f34000ae370 2024-03-28T01:06:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.023+0000 7f3405200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34000ae910 con 0x7f340014b720 2024-03-28T01:06:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.023+0000 7f3405200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000aebf0 con 0x7f340006d3e0 2024-03-28T01:06:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.023+0000 7f78b5000640 1 Processor -- start 2024-03-28T01:06:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.023+0000 7f78b5000640 1 -- start start 2024-03-28T01:06:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f1dae000640 1 -- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1db01d9230 msgr2=0x7f1db01d96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f1dae000640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1db01d9230 0x7f1db01d96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f1dae000640 1 -- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db0066dd0 msgr2=0x7f1db009dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f1dae000640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db0066dd0 0x7f1db009dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f1dae000640 1 -- 172.21.15.33:0/3984323615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db01d9c00 con 0x7f1db00b5690 2024-03-28T01:06:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f33fe800640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7f340014b720 0x7f340014bb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37950/0 (socket says 172.21.15.33:37950) 2024-03-28T01:06:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f33fe800640 1 -- 172.21.15.33:0/1545831942 learned_addr learned my addr 172.21.15.33:0/1545831942 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.024+0000 7f33fde00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340006d3e0 0x7f34000af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f1daea00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db0066dd0 0x7f1db009dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:32.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7fe506800640 1 Processor -- start 2024-03-28T01:06:32.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f78b5000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78b006d170 0x7f78b014f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f78b5000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f78b5000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b00ae880 con 0x7f78b00ae4a0 2024-03-28T01:06:32.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f78b5000640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b00afbd0 con 0x7f78b006d170 2024-03-28T01:06:32.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f78b5000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00b01e0 con 0x7f78b006d600 2024-03-28T01:06:32.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f33fca00640 1 -- 172.21.15.33:0/1545831942 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790567407 0 0) 0x7f34000ae910 con 0x7f340014b720 2024-03-28T01:06:32.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7fe506800640 1 -- start start 2024-03-28T01:06:32.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.025+0000 7f1dae000640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db00b5690 0x7f1db00b2fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1da00098c0 tx=0x7f1da00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.026+0000 7f33fca00640 1 -- 172.21.15.33:0/1545831942 --> v1:172.21.15.194:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e4003520 con 0x7f340014b720 2024-03-28T01:06:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.026+0000 7f33fde00640 1 -- 172.21.15.33:0/1545831942 >> v1:172.21.15.194:6789/0 conn(0x7f340014b720 legacy=0x7f340014bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.026+0000 7f33fde00640 1 -- 172.21.15.33:0/1545831942 >> v1:172.21.15.33:6789/0 conn(0x7f34000ae370 legacy=0x7f34000b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.026+0000 7f78ade00640 1 --1- >> v1:172.21.15.194:6789/0 conn(0x7f78b00ae4a0 0x7f78b00af4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.194:6789/0 says I am v1:172.21.15.33:37964/0 (socket says 172.21.15.33:37964) 2024-03-28T01:06:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.026+0000 7f78ade00640 1 -- 172.21.15.33:0/845555 learned_addr learned my addr 172.21.15.33:0/845555 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.026+0000 7f78ae800640 1 --2- 172.21.15.33:0/845555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.026+0000 7f33fde00640 1 -- 172.21.15.33:0/1545831942 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000aeed0 con 0x7f340006d3e0 2024-03-28T01:06:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f78aca00640 1 -- 172.21.15.33:0/845555 <== mon.1 v1:172.21.15.194:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958921145 0 0) 0x7f78b00ae880 con 0x7f78b00ae4a0 2024-03-28T01:06:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f78aca00640 1 -- 172.21.15.33:0/845555 --> v1:172.21.15.194:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7894003520 con 0x7f78b00ae4a0 2024-03-28T01:06:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f78ae800640 1 -- 172.21.15.33:0/845555 >> v2:172.21.15.33:3300/0 conn(0x7f78b006d170 msgr2=0x7f78b014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f78ae800640 1 --2- 172.21.15.33:0/845555 >> v2:172.21.15.33:3300/0 conn(0x7f78b006d170 0x7f78b014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f78ae800640 1 -- 172.21.15.33:0/845555 >> v1:172.21.15.194:6789/0 conn(0x7f78b00ae4a0 legacy=0x7f78b00af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f78ae800640 1 -- 172.21.15.33:0/845555 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00aeb30 con 0x7f78b006d600 2024-03-28T01:06:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f1da7400640 1 -- 172.21.15.33:0/3984323615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0040020 con 0x7f1db00b5690 2024-03-28T01:06:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f1da7400640 1 -- 172.21.15.33:0/3984323615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0032a10 con 0x7f1db00b5690 2024-03-28T01:06:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f1da7400640 1 -- 172.21.15.33:0/3984323615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0037040 con 0x7f1db00b5690 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7fe506800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe5000b5830 0x7fe5000b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.028+0000 7fe506800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe50014af40 0x7fe5001413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.028+0000 7fe506800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5000b5c10 con 0x7fe500140f50 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.028+0000 7fe506800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe500141920 con 0x7fe5000b5830 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.028+0000 7fe506800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000b6a90 con 0x7fe50014af40 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.029+0000 7fe4ffe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe50014af40 0x7fe5001413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.029+0000 7fe4ffe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe50014af40 0x7fe5001413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:48248/0 (socket says 172.21.15.33:48248) 2024-03-28T01:06:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.029+0000 7fe4ffe00640 1 -- 172.21.15.33:0/1379956164 learned_addr learned my addr 172.21.15.33:0/1379956164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.027+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db009e570 con 0x7f1db00b5690 2024-03-28T01:06:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.028+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db009ebc0 con 0x7f1db00b5690 2024-03-28T01:06:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.029+0000 7fe4ff400640 1 --2- 172.21.15.33:0/1379956164 >> v2:172.21.15.33:3300/0 conn(0x7fe5000b5830 0x7fe5000b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.029+0000 7f33fde00640 1 --2- 172.21.15.33:0/1545831942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340006d3e0 0x7f34000af680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0097d0 tx=0x7f33ec02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee9b5374f49ec5b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.030+0000 7fe4ff400640 1 -- 172.21.15.33:0/1379956164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe50014af40 msgr2=0x7fe5001413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.030+0000 7fe4ff400640 1 --2- 172.21.15.33:0/1379956164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe50014af40 0x7fe5001413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.030+0000 7fe4ff400640 1 -- 172.21.15.33:0/1379956164 >> v1:172.21.15.194:6789/0 conn(0x7fe500140f50 legacy=0x7fe50014a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.030+0000 7fe4ff400640 1 -- 172.21.15.33:0/1379956164 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000b4290 con 0x7fe5000b5830 2024-03-28T01:06:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.030+0000 7f33fca00640 1 -- 172.21.15.33:0/1545831942 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec031770 con 0x7f340006d3e0 2024-03-28T01:06:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.031+0000 7f33fca00640 1 -- 172.21.15.33:0/1545831942 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec037070 con 0x7f340006d3e0 2024-03-28T01:06:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.031+0000 7f33fca00640 1 -- 172.21.15.33:0/1545831942 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec031b30 con 0x7f340006d3e0 2024-03-28T01:06:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.030+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db0075380 con 0x7f1db00b5690 2024-03-28T01:06:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.031+0000 7f78ae800640 1 --2- 172.21.15.33:0/845555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b014f9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009d90 tx=0x7f78a802bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef82390adf04bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7f78aca00640 1 -- 172.21.15.33:0/845555 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8039070 con 0x7f78b006d600 2024-03-28T01:06:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7fe4ff400640 1 --2- 172.21.15.33:0/1379956164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5000b5830 0x7fe5000b9d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00097d0 tx=0x7fe4f002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05b6d61436944c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7f3405200640 1 -- 172.21.15.33:0/1545831942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340006d3e0 msgr2=0x7f34000af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7fe4fe000640 1 -- 172.21.15.33:0/1379956164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0031770 con 0x7fe5000b5830 2024-03-28T01:06:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7f78aca00640 1 -- 172.21.15.33:0/845555 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a802cb30 con 0x7f78b006d600 2024-03-28T01:06:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7f78aca00640 1 -- 172.21.15.33:0/845555 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8034440 con 0x7f78b006d600 2024-03-28T01:06:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7fe4fe000640 1 -- 172.21.15.33:0/1379956164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0037070 con 0x7fe5000b5830 2024-03-28T01:06:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.032+0000 7fe4fe000640 1 -- 172.21.15.33:0/1379956164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0031b30 con 0x7fe5000b5830 2024-03-28T01:06:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.033+0000 7f3405200640 1 --2- 172.21.15.33:0/1545831942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340006d3e0 0x7f34000af680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0097d0 tx=0x7f33ec02b810 comp rx=0 tx=0).stop 2024-03-28T01:06:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.033+0000 7f3405200640 1 -- 172.21.15.33:0/1545831942 shutdown_connections 2024-03-28T01:06:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.033+0000 7f3405200640 1 --2- 172.21.15.33:0/1545831942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340006d3e0 0x7f34000af680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.033+0000 7fe506800640 1 -- 172.21.15.33:0/1379956164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5000b5830 msgr2=0x7fe5000b9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.033+0000 7fe506800640 1 --2- 172.21.15.33:0/1379956164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5000b5830 0x7fe5000b9d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00097d0 tx=0x7fe4f002b810 comp rx=0 tx=0).stop 2024-03-28T01:06:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.034+0000 7f3405200640 1 -- 172.21.15.33:0/1545831942 >> 172.21.15.33:0/1545831942 conn(0x7f340006b030 msgr2=0x7f340006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.034+0000 7fe506800640 1 -- 172.21.15.33:0/1379956164 shutdown_connections 2024-03-28T01:06:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.034+0000 7fe506800640 1 --2- 172.21.15.33:0/1379956164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe50014af40 0x7fe5001413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.034+0000 7fe506800640 1 --2- 172.21.15.33:0/1379956164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5000b5830 0x7fe5000b9d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.034+0000 7fe506800640 1 -- 172.21.15.33:0/1379956164 >> 172.21.15.33:0/1379956164 conn(0x7fe500072b80 msgr2=0x7fe500072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.033+0000 7f1da7400640 1 -- 172.21.15.33:0/3984323615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1da0043020 con 0x7f1db00b5690 2024-03-28T01:06:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f3405200640 1 -- 172.21.15.33:0/1545831942 shutdown_connections 2024-03-28T01:06:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f3405200640 1 -- 172.21.15.33:0/1545831942 wait complete. 2024-03-28T01:06:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f1da7400640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1d9006bca0 0x7f1d9006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f1da7400640 1 -- 172.21.15.33:0/3984323615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1da00ac0d0 con 0x7f1db00b5690 2024-03-28T01:06:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.036+0000 7fe506800640 1 -- 172.21.15.33:0/1379956164 shutdown_connections 2024-03-28T01:06:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.034+0000 7f78b5000640 1 -- 172.21.15.33:0/845555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 msgr2=0x7f78b014f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.034+0000 7f78b5000640 1 --2- 172.21.15.33:0/845555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b014f9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009d90 tx=0x7f78a802bd80 comp rx=0 tx=0).stop 2024-03-28T01:06:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f78b5000640 1 -- 172.21.15.33:0/845555 shutdown_connections 2024-03-28T01:06:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f78b5000640 1 --2- 172.21.15.33:0/845555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b014f9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f78b5000640 1 --2- 172.21.15.33:0/845555 >> v2:172.21.15.33:3300/0 conn(0x7f78b006d170 0x7f78b014f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.035+0000 7f78b5000640 1 -- 172.21.15.33:0/845555 >> 172.21.15.33:0/845555 conn(0x7f78b00b69d0 msgr2=0x7f78b00b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.037+0000 7f3405200640 1 Processor -- start 2024-03-28T01:06:32.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.036+0000 7f78b5000640 1 -- 172.21.15.33:0/845555 shutdown_connections 2024-03-28T01:06:32.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.036+0000 7f1daea00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1d9006bca0 0x7f1d9006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.038+0000 7f1daea00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1d9006bca0 0x7f1d9006e160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f60 tx=0x7f1d9c007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.038+0000 7f3405200640 1 -- start start 2024-03-28T01:06:32.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.036+0000 7fe506800640 1 -- 172.21.15.33:0/1379956164 wait complete. 2024-03-28T01:06:32.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.038+0000 7fe506800640 1 Processor -- start 2024-03-28T01:06:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.038+0000 7fe506800640 1 -- start start 2024-03-28T01:06:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7fe506800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5000b5830 0x7fe5001d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.038+0000 7f78b5000640 1 -- 172.21.15.33:0/845555 wait complete. 2024-03-28T01:06:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7fe506800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe500140f50 0x7fe5001472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe506800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe50014af40 0x7fe500147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7f78b5000640 1 Processor -- start 2024-03-28T01:06:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe506800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000ba280 con 0x7fe50014af40 2024-03-28T01:06:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe4ff400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5000b5830 0x7fe5001d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe506800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000b4850 con 0x7fe500140f50 2024-03-28T01:06:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe506800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000ba400 con 0x7fe5000b5830 2024-03-28T01:06:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe4fea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe500140f50 0x7fe5001472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe4fea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe500140f50 0x7fe5001472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44572/0 (socket says 172.21.15.33:44572) 2024-03-28T01:06:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7fe4fea00640 1 -- 172.21.15.33:0/2964837796 learned_addr learned my addr 172.21.15.33:0/2964837796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.041+0000 7f78b5000640 1 -- start start 2024-03-28T01:06:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7fe4ff400640 1 -- 172.21.15.33:0/2964837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe500140f50 msgr2=0x7fe5001472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7fe4ff400640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe500140f50 0x7fe5001472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7fe4ff400640 1 -- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe50014af40 msgr2=0x7fe500147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7fe4ff400640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe50014af40 0x7fe500147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7f3405200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f340006d3e0 0x7f34001e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7f3405200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34000ae370 0x7f34001e1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.043+0000 7fe4ff400640 1 -- 172.21.15.33:0/2964837796 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5001d2690 con 0x7fe5000b5830 2024-03-28T01:06:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.043+0000 7f1da7400640 1 -- 172.21.15.33:0/3984323615 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f1da004a030 con 0x7f1db00b5690 2024-03-28T01:06:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.043+0000 7f78b5000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78b006d170 0x7f78b014d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.043+0000 7f78b5000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b01483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.043+0000 7f78b5000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b00ae4a0 0x7f78b0148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7fe4ff400640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5000b5830 0x7fe5001d2e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5000ba7d0 tx=0x7fe4f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7f3405200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340014b720 0x7f34001e1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f78b5000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00ade50 con 0x7f78b006d170 2024-03-28T01:06:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f78b5000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00af1e0 con 0x7f78b00ae4a0 2024-03-28T01:06:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f78b5000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0156370 con 0x7f78b006d600 2024-03-28T01:06:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7f3405200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340014c400 con 0x7f340006d3e0 2024-03-28T01:06:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f78ade00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78b006d170 0x7f78b014d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f78ade00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78b006d170 0x7f78b014d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36618/0 (socket says 172.21.15.33:36618) 2024-03-28T01:06:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f78ade00640 1 -- 172.21.15.33:0/1134291301 learned_addr learned my addr 172.21.15.33:0/1134291301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7f3405200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340014c280 con 0x7f34000ae370 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.040+0000 7f3405200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340014c580 con 0x7f340014b720 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7f33fd400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34000ae370 0x7f34001e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7f33fd400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34000ae370 0x7f34001e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44562/0 (socket says 172.21.15.33:44562) 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7f33fd400640 1 -- 172.21.15.33:0/2246751460 learned_addr learned my addr 172.21.15.33:0/2246751460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.042+0000 7f33fd400640 1 -- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340014b720 msgr2=0x7f34001e1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f33fde00640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f340006d3e0 0x7f34001e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7f78ad400640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b01483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.045+0000 7f78ae800640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b00ae4a0 0x7f78b0148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7fe4f7400640 1 -- 172.21.15.33:0/2964837796 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0045600 con 0x7fe5000b5830 2024-03-28T01:06:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7fe4f7400640 1 -- 172.21.15.33:0/2964837796 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f00457c0 con 0x7fe5000b5830 2024-03-28T01:06:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7fe4f7400640 1 -- 172.21.15.33:0/2964837796 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0045bb0 con 0x7fe5000b5830 2024-03-28T01:06:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.044+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5001d05e0 con 0x7fe5000b5830 2024-03-28T01:06:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.045+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5001d23b0 con 0x7fe5000b5830 2024-03-28T01:06:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f78ade00640 1 -- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 msgr2=0x7f78b01483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f78ade00640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b01483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f78ade00640 1 -- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b00ae4a0 msgr2=0x7f78b0148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f78ade00640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b00ae4a0 0x7f78b0148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f78ade00640 1 -- 172.21.15.33:0/1134291301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b009b040 con 0x7f78b006d170 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.043+0000 7f33fe800640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340014b720 0x7f34001e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f33fd400640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340014b720 0x7f34001e1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f33fd400640 1 -- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f340006d3e0 msgr2=0x7f34001e4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f33fd400640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f340006d3e0 0x7f34001e4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f33fd400640 1 -- 172.21.15.33:0/2246751460 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34001409d0 con 0x7f34000ae370 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f33fde00640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f340006d3e0 0x7f34001e4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7f33fe800640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340014b720 0x7f34001e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f33fd400640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34000ae370 0x7f34001e1240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f33f000c870 tx=0x7f33f000cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f78ade00640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78b006d170 0x7f78b014d520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f789800c410 tx=0x7f789800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.046+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50014ad60 con 0x7fe5000b5830 2024-03-28T01:06:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f78a6000640 1 -- 172.21.15.33:0/1134291301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898017070 con 0x7f78b006d170 2024-03-28T01:06:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f78a6000640 1 -- 172.21.15.33:0/1134291301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7898005ce0 con 0x7f78b006d170 2024-03-28T01:06:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f78a6000640 1 -- 172.21.15.33:0/1134291301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980072f0 con 0x7f78b006d170 2024-03-28T01:06:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f33f6000640 1 -- 172.21.15.33:0/2246751460 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0003b70 con 0x7f34000ae370 2024-03-28T01:06:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f33f6000640 1 -- 172.21.15.33:0/2246751460 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f0003d30 con 0x7f34000ae370 2024-03-28T01:06:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.047+0000 7f33f6000640 1 -- 172.21.15.33:0/2246751460 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000aa90 con 0x7f34000ae370 2024-03-28T01:06:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.049+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b01ed520 con 0x7f78b006d170 2024-03-28T01:06:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.049+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b014e6a0 con 0x7f78b006d170 2024-03-28T01:06:32.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.049+0000 7fe4f7400640 1 -- 172.21.15.33:0/2964837796 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe4f0031070 con 0x7fe5000b5830 2024-03-28T01:06:32.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.048+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3400140c40 con 0x7f34000ae370 2024-03-28T01:06:32.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.048+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34001e4f60 con 0x7f34000ae370 2024-03-28T01:06:32.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.050+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3400068a90 con 0x7f34000ae370 2024-03-28T01:06:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.051+0000 7fe4f7400640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe4e006bc20 0x7fe4e006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.051+0000 7fe4f7400640 1 -- 172.21.15.33:0/2964837796 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4f00ac420 con 0x7fe5000b5830 2024-03-28T01:06:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.051+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b0075380 con 0x7f78b006d170 2024-03-28T01:06:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.052+0000 7f33f6000640 1 -- 172.21.15.33:0/2246751460 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f33f000ac30 con 0x7f34000ae370 2024-03-28T01:06:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.054+0000 7f33f6000640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f33e006bca0 0x7f33e006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.054+0000 7f33f6000640 1 -- 172.21.15.33:0/2246751460 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33f008b120 con 0x7f34000ae370 2024-03-28T01:06:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.055+0000 7f33fde00640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f33e006bca0 0x7f33e006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.056+0000 7f33fde00640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f33e006bca0 0x7f33e006e160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33ec036000 tx=0x7f33ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.056+0000 7fe4fea00640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe4e006bc20 0x7fe4e006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.057+0000 7f78a6000640 1 -- 172.21.15.33:0/1134291301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f789801a070 con 0x7f78b006d170 2024-03-28T01:06:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.058+0000 7f78a6000640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f788c06bca0 0x7f788c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.058+0000 7f78a6000640 1 -- 172.21.15.33:0/1134291301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f789808a3c0 con 0x7f78b006d170 2024-03-28T01:06:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.059+0000 7f78ad400640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f788c06bca0 0x7f788c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.060+0000 7f78ad400640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f788c06bca0 0x7f788c06e160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78b014e0b0 tx=0x7f78a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.061+0000 7fe4fea00640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe4e006bc20 0x7fe4e006e0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe500141540 tx=0x7fe4ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.062+0000 7fe4f7400640 1 -- 172.21.15.33:0/2964837796 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe50014ad60 con 0x7fe5000b5830 2024-03-28T01:06:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.064+0000 7f78a6000640 1 -- 172.21.15.33:0/1134291301 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f7898058900 con 0x7f78b006d170 2024-03-28T01:06:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.065+0000 7f33f6000640 1 -- 172.21.15.33:0/2246751460 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3400068a90 con 0x7f34000ae370 2024-03-28T01:06:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.087+0000 7f7b39800640 1 Processor -- start 2024-03-28T01:06:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.087+0000 7f7b39800640 1 -- start start 2024-03-28T01:06:32.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.089+0000 7f7b39800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b5fe0 0x7f7b340b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.089+0000 7f7b39800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b340b3a60 con 0x7f7b340b63c0 2024-03-28T01:06:32.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.089+0000 7f7b39800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3414bbe0 con 0x7f7b3414f2d0 2024-03-28T01:06:32.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.089+0000 7f7b39800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34145e30 con 0x7f7b340b5fe0 2024-03-28T01:06:32.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.089+0000 7f7b33400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7b340b63c0 0x7f7b3414ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41146/0 (socket says 172.21.15.33:41146) 2024-03-28T01:06:32.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.089+0000 7f7b33400640 1 -- 172.21.15.33:0/1109914075 learned_addr learned my addr 172.21.15.33:0/1109914075 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:32.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.090+0000 7f7b33e00640 1 --2- 172.21.15.33:0/1109914075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b5fe0 0x7f7b340b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.090+0000 7f7b33e00640 1 -- 172.21.15.33:0/1109914075 >> v1:172.21.15.194:6789/0 conn(0x7f7b3414f2d0 legacy=0x7f7b34155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:32.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.090+0000 7f7b33e00640 1 -- 172.21.15.33:0/1109914075 >> v1:172.21.15.33:6789/0 conn(0x7f7b340b63c0 legacy=0x7f7b3414ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.090+0000 7f7b33e00640 1 -- 172.21.15.33:0/1109914075 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3414bec0 con 0x7f7b340b5fe0 2024-03-28T01:06:32.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.092+0000 7f7b33e00640 1 --2- 172.21.15.33:0/1109914075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b5fe0 0x7f7b340b9670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b240097d0 tx=0x7f7b2402b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd4e0a2ee7dae8d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.093+0000 7f7b32a00640 1 -- 172.21.15.33:0/1109914075 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24037070 con 0x7f7b340b5fe0 2024-03-28T01:06:32.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.094+0000 7f7b32a00640 1 -- 172.21.15.33:0/1109914075 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24030710 con 0x7f7b340b5fe0 2024-03-28T01:06:32.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.094+0000 7f7b32a00640 1 -- 172.21.15.33:0/1109914075 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24037070 con 0x7f7b340b5fe0 2024-03-28T01:06:32.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.095+0000 7f7b39800640 1 -- 172.21.15.33:0/1109914075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b5fe0 msgr2=0x7f7b340b9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.095+0000 7f7b39800640 1 --2- 172.21.15.33:0/1109914075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b5fe0 0x7f7b340b9670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b240097d0 tx=0x7f7b2402b6e0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.096+0000 7f7b39800640 1 -- 172.21.15.33:0/1109914075 shutdown_connections 2024-03-28T01:06:32.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.096+0000 7f7b39800640 1 --2- 172.21.15.33:0/1109914075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b5fe0 0x7f7b340b9670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.096+0000 7f7b39800640 1 -- 172.21.15.33:0/1109914075 >> 172.21.15.33:0/1109914075 conn(0x7f7b34072940 msgr2=0x7f7b34072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.096+0000 7f1c52000640 1 Processor -- start 2024-03-28T01:06:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.096+0000 7f1c52000640 1 -- start start 2024-03-28T01:06:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.098+0000 7f1c52000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f1c4c0b9cc0 0x7f1c4c145630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.098+0000 7f1c52000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c145b70 0x7f1c4c14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.098+0000 7f1c52000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c14f870 con 0x7f1c4c14f3e0 2024-03-28T01:06:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.098+0000 7f1c52000640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b2d50 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.098+0000 7f1c52000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0ba0a0 con 0x7f1c4c145b70 2024-03-28T01:06:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.098+0000 7f1c4aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c145b70 0x7f1c4c14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.099+0000 7f1c4be00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1c4c14f3e0 0x7f1c4c0b75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41156/0 (socket says 172.21.15.33:41156) 2024-03-28T01:06:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.099+0000 7f1c4be00640 1 -- 172.21.15.33:0/1269014263 learned_addr learned my addr 172.21.15.33:0/1269014263 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.099+0000 7f1c4aa00640 1 -- 172.21.15.33:0/1269014263 >> v2:172.21.15.194:3300/0 conn(0x7f1c4c0b9cc0 msgr2=0x7f1c4c145630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-28T01:06:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.099+0000 7f1c4aa00640 1 --2- 172.21.15.33:0/1269014263 >> v2:172.21.15.194:3300/0 conn(0x7f1c4c0b9cc0 0x7f1c4c145630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.099+0000 7f1c4aa00640 1 -- 172.21.15.33:0/1269014263 >> v1:172.21.15.33:6789/0 conn(0x7f1c4c14f3e0 legacy=0x7f1c4c0b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.099+0000 7f1c4aa00640 1 -- 172.21.15.33:0/1269014263 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0b2fa0 con 0x7f1c4c145b70 2024-03-28T01:06:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.101+0000 7f7b39800640 1 -- 172.21.15.33:0/1109914075 shutdown_connections 2024-03-28T01:06:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.101+0000 7f7b39800640 1 -- 172.21.15.33:0/1109914075 wait complete. 2024-03-28T01:06:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.102+0000 7f1c4aa00640 1 --2- 172.21.15.33:0/1269014263 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c145b70 0x7f1c4c14eea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0097a0 tx=0x7f1c3c02b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4715979c98d4fd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.103+0000 7f7b39800640 1 Processor -- start 2024-03-28T01:06:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.103+0000 7f1c4a000640 1 -- 172.21.15.33:0/1269014263 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c038070 con 0x7f1c4c145b70 2024-03-28T01:06:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.103+0000 7f1c4a000640 1 -- 172.21.15.33:0/1269014263 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c031710 con 0x7f1c4c145b70 2024-03-28T01:06:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.103+0000 7f1c4a000640 1 -- 172.21.15.33:0/1269014263 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c031ae0 con 0x7f1c4c145b70 2024-03-28T01:06:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.104+0000 7f1c52000640 1 -- 172.21.15.33:0/1269014263 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c145b70 msgr2=0x7f1c4c14eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.104+0000 7f1c52000640 1 --2- 172.21.15.33:0/1269014263 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c145b70 0x7f1c4c14eea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0097a0 tx=0x7f1c3c02b7b0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.103+0000 7f7b39800640 1 -- start start 2024-03-28T01:06:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.104+0000 7f7b39800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b340b5fe0 0x7f7b34148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f1c52000640 1 -- 172.21.15.33:0/1269014263 shutdown_connections 2024-03-28T01:06:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f1c52000640 1 --2- 172.21.15.33:0/1269014263 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c145b70 0x7f1c4c14eea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f1c52000640 1 --2- 172.21.15.33:0/1269014263 >> v2:172.21.15.194:3300/0 conn(0x7f1c4c0b9cc0 0x7f1c4c145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f1c52000640 1 -- 172.21.15.33:0/1269014263 >> 172.21.15.33:0/1269014263 conn(0x7f1c4c072940 msgr2=0x7f1c4c072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f1c52000640 1 -- 172.21.15.33:0/1269014263 shutdown_connections 2024-03-28T01:06:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f1c52000640 1 -- 172.21.15.33:0/1269014263 wait complete. 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.104+0000 7f7b39800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b63c0 0x7f7b341472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.104+0000 7f7b39800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b340bb7e0 0x7f7b34147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f7b39800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340b6c90 con 0x7f7b340bb7e0 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f7b39800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340b36a0 con 0x7f7b340b5fe0 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f7b39800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340b6e10 con 0x7f7b340b63c0 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f7b33e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b340b5fe0 0x7f7b34148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.105+0000 7f7b33e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b340b5fe0 0x7f7b34148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:44600/0 (socket says 172.21.15.33:44600) 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.106+0000 7f7b33e00640 1 -- 172.21.15.33:0/3948786510 learned_addr learned my addr 172.21.15.33:0/3948786510 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.106+0000 7f7b33400640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b63c0 0x7f7b341472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.106+0000 7f7b33e00640 1 -- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b63c0 msgr2=0x7f7b341472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.106+0000 7f7b33e00640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b63c0 0x7f7b341472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.106+0000 7f7b33e00640 1 -- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b340bb7e0 msgr2=0x7f7b34147eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T01:06:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.106+0000 7f7b33e00640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b340bb7e0 0x7f7b34147eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.106+0000 7f7b33e00640 1 -- 172.21.15.33:0/3948786510 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34147830 con 0x7f7b340b5fe0 2024-03-28T01:06:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.107+0000 7f7b33e00640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b340b5fe0 0x7f7b34148c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b24000c00 tx=0x7f7b240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.107+0000 7f7b2be00640 1 -- 172.21.15.33:0/3948786510 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24042020 con 0x7f7b340b5fe0 2024-03-28T01:06:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.108+0000 7f1c52000640 1 Processor -- start 2024-03-28T01:06:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.108+0000 7f7b2be00640 1 -- 172.21.15.33:0/3948786510 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24003610 con 0x7f7b340b5fe0 2024-03-28T01:06:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.108+0000 7f7b2be00640 1 -- 172.21.15.33:0/3948786510 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24037040 con 0x7f7b340b5fe0 2024-03-28T01:06:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.108+0000 7f1c52000640 1 -- start start 2024-03-28T01:06:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.109+0000 7f1c52000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c4c0b9cc0 0x7f1c4c149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.109+0000 7f1c52000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c4c145b70 0x7f1c4c1483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.109+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b341d4a00 con 0x7f7b340b5fe0 2024-03-28T01:06:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.110+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b34146af0 con 0x7f7b340b5fe0 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.110+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c4c0b9cc0 0x7f1c4c149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.110+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c4c0b9cc0 0x7f1c4c149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36630/0 (socket says 172.21.15.33:36630) 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.110+0000 7f1c4b400640 1 -- 172.21.15.33:0/1187551999 learned_addr learned my addr 172.21.15.33:0/1187551999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.110+0000 7f1c52000640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c14f3e0 0x7f1c4c14a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.111+0000 7f1c4aa00640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c4c145b70 0x7f1c4c1483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.111+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b9410 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.111+0000 7f1c4be00640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c14f3e0 0x7f1c4c14a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.111+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b3560 con 0x7f1c4c145b70 2024-03-28T01:06:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.111+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b34075310 con 0x7f7b340b5fe0 2024-03-28T01:06:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.111+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b8690 con 0x7f1c4c14f3e0 2024-03-28T01:06:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.113+0000 7f1c4b400640 1 -- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c14f3e0 msgr2=0x7f1c4c14a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.113+0000 7f1c4b400640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c14f3e0 0x7f1c4c14a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.113+0000 7f1c4b400640 1 -- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c4c145b70 msgr2=0x7f1c4c1483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.113+0000 7f1c4b400640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c4c145b70 0x7f1c4c1483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.113+0000 7f1c4b400640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c14c2e0 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.114+0000 7f7b2be00640 1 -- 172.21.15.33:0/3948786510 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7b24045020 con 0x7f7b340b5fe0 2024-03-28T01:06:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.114+0000 7f1c4b400640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c4c0b9cc0 0x7f1c4c149750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800b7b0 tx=0x7f1c3800bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.115+0000 7f1c43400640 1 -- 172.21.15.33:0/1187551999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38019070 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.115+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c12fc70 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.115+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c13f4a0 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.115+0000 7f1c43400640 1 -- 172.21.15.33:0/1187551999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800c720 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.116+0000 7f7b2be00640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7b1406bbd0 0x7f7b1406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.116+0000 7f7b2be00640 1 -- 172.21.15.33:0/3948786510 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b240acb10 con 0x7f7b340b5fe0 2024-03-28T01:06:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.116+0000 7f7b33400640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7b1406bbd0 0x7f7b1406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.116+0000 7f1c43400640 1 -- 172.21.15.33:0/1187551999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38019070 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.119+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c24005290 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.119+0000 7f7b33400640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7b1406bbd0 0x7f7b1406e090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b341416e0 tx=0x7f7b20007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.120+0000 7f1c43400640 1 -- 172.21.15.33:0/1187551999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1c3800c8d0 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.122+0000 7f1c43400640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1c2c06c000 0x7f1c2c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.123+0000 7f1c4aa00640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1c2c06c000 0x7f1c2c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.124+0000 7f1c4aa00640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1c2c06c000 0x7f1c2c06e4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c035000 tx=0x7f1c3c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.124+0000 7f1c43400640 1 -- 172.21.15.33:0/1187551999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c3800c520 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.132+0000 7f1c43400640 1 -- 172.21.15.33:0/1187551999 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f1c380b62e0 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.133+0000 7f7b2be00640 1 -- 172.21.15.33:0/3948786510 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f7b2404c030 con 0x7f7b340b5fe0 2024-03-28T01:06:32.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.416+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb1680af360 con 0x7fb168147620 2024-03-28T01:06:32.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.419+0000 7fb15f400640 1 -- 172.21.15.33:0/4168850860 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb15407a340 con 0x7fb168147620 2024-03-28T01:06:32.422 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:32.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.423+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb14806bdd0 msgr2=0x7fb14806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.423+0000 7fb16e200640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb14806bdd0 0x7fb14806e290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb1680b4e80 tx=0x7fb1600074c0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb168147620 msgr2=0x7fb16806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb168147620 0x7fb16806fbc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb154039040 tx=0x7fb1540046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 shutdown_connections 2024-03-28T01:06:32.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fb14806bdd0 0x7fb14806e290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb168147620 0x7fb16806fbc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1681471e0 0x7fb16806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 --2- 172.21.15.33:0/4168850860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb168146e00 0x7fb1680b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.424+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 >> 172.21.15.33:0/4168850860 conn(0x7fb1680b69d0 msgr2=0x7fb1681ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.425+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 shutdown_connections 2024-03-28T01:06:32.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.426+0000 7fb16e200640 1 -- 172.21.15.33:0/4168850860 wait complete. 2024-03-28T01:06:32.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-03-28T01:06:32.454 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:32.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.480+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78b00af770 con 0x7f78b006d170 2024-03-28T01:06:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.481+0000 7f78a6000640 1 -- 172.21.15.33:0/1134291301 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f789805c9a0 con 0x7f78b006d170 2024-03-28T01:06:32.484 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-03-28T01:06:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f788c06bca0 msgr2=0x7f788c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f788c06bca0 0x7f788c06e160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78b014e0b0 tx=0x7f78a0008040 comp rx=0 tx=0).stop 2024-03-28T01:06:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78b006d170 msgr2=0x7f78b014d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78b006d170 0x7f78b014d520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f789800c410 tx=0x7f789800c8d0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 shutdown_connections 2024-03-28T01:06:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f788c06bca0 0x7f788c06e160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b00ae4a0 0x7f78b0148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78b006d600 0x7f78b01483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 --2- 172.21.15.33:0/1134291301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78b006d170 0x7f78b014d520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.484+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 >> 172.21.15.33:0/1134291301 conn(0x7f78b00b69d0 msgr2=0x7f78b00b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.485+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 shutdown_connections 2024-03-28T01:06:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.485+0000 7f78b5000640 1 -- 172.21.15.33:0/1134291301 wait complete. 2024-03-28T01:06:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.497+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe500149310 con 0x7fe5000b5830 2024-03-28T01:06:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.498+0000 7fe4f7400640 1 -- 172.21.15.33:0/2964837796 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe500149310 con 0x7fe5000b5830 2024-03-28T01:06:32.501 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.499+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1db013cda0 con 0x7f1db00b5690 2024-03-28T01:06:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.500+0000 7f1da7400640 1 -- 172.21.15.33:0/3984323615 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1da007ac30 con 0x7f1db00b5690 2024-03-28T01:06:32.503 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:32.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe4e006bc20 msgr2=0x7fe4e006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe4e006bc20 0x7fe4e006e0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe500141540 tx=0x7fe4ec009040 comp rx=0 tx=0).stop 2024-03-28T01:06:32.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5000b5830 msgr2=0x7fe5001d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5000b5830 0x7fe5001d2e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5000ba7d0 tx=0x7fe4f00046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 shutdown_connections 2024-03-28T01:06:32.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fe4e006bc20 0x7fe4e006e0e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe50014af40 0x7fe500147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe500140f50 0x7fe5001472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 --2- 172.21.15.33:0/2964837796 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5000b5830 0x7fe5001d2e20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.501+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 >> 172.21.15.33:0/2964837796 conn(0x7fe500072b80 msgr2=0x7fe5000b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.502+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 shutdown_connections 2024-03-28T01:06:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.502+0000 7fe506800640 1 -- 172.21.15.33:0/2964837796 wait complete. 2024-03-28T01:06:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.503+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1c24005850 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1c43400640 1 -- 172.21.15.33:0/1187551999 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c38058360 con 0x7f1c4c0b9cc0 2024-03-28T01:06:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1d9006bca0 msgr2=0x7f1d9006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1d9006bca0 0x7f1d9006e160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f60 tx=0x7f1d9c007ce0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db00b5690 msgr2=0x7f1db00b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db00b5690 0x7f1db00b2fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1da00098c0 tx=0x7f1da00046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.507 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 shutdown_connections 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1d9006bca0 0x7f1d9006e160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1db01d9230 0x7f1db01d96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db00b5690 0x7f1db00b2fb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 --2- 172.21.15.33:0/3984323615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db0066dd0 0x7f1db009dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.504+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 >> 172.21.15.33:0/3984323615 conn(0x7f1db00b9800 msgr2=0x7f1db009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.505+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 shutdown_connections 2024-03-28T01:06:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.505+0000 7f1db5e00640 1 -- 172.21.15.33:0/3984323615 wait complete. 2024-03-28T01:06:32.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-03-28T01:06:32.509 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:32.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.506+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1c2c06c000 msgr2=0x7f1c2c06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.506+0000 7f1c52000640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1c2c06c000 0x7f1c2c06e4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c035000 tx=0x7f1c3c008030 comp rx=0 tx=0).stop 2024-03-28T01:06:32.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.506+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c4c0b9cc0 msgr2=0x7f1c4c149750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.506+0000 7f1c52000640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c4c0b9cc0 0x7f1c4c149750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800b7b0 tx=0x7f1c3800bc70 comp rx=0 tx=0).stop 2024-03-28T01:06:32.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 shutdown_connections 2024-03-28T01:06:32.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1c2c06c000 0x7f1c2c06e4c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c4c14f3e0 0x7f1c4c14a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c4c145b70 0x7f1c4c1483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 --2- 172.21.15.33:0/1187551999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c4c0b9cc0 0x7f1c4c149750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 >> 172.21.15.33:0/1187551999 conn(0x7f1c4c072940 msgr2=0x7f1c4c143220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 shutdown_connections 2024-03-28T01:06:32.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.507+0000 7f1c52000640 1 -- 172.21.15.33:0/1187551999 wait complete. 2024-03-28T01:06:32.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.513+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3400075380 con 0x7f34000ae370 2024-03-28T01:06:32.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.514+0000 7f33f6000640 1 -- 172.21.15.33:0/2246751460 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f33f0059bf0 con 0x7f34000ae370 2024-03-28T01:06:32.517 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2024-03-28T01:06:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f33e006bca0 msgr2=0x7f33e006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f33e006bca0 0x7f33e006e160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33ec036000 tx=0x7f33ec008030 comp rx=0 tx=0).stop 2024-03-28T01:06:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34000ae370 msgr2=0x7f34001e1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34000ae370 0x7f34001e1240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f33f000c870 tx=0x7f33f000cd30 comp rx=0 tx=0).stop 2024-03-28T01:06:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 shutdown_connections 2024-03-28T01:06:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f33e006bca0 0x7f33e006e160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f340014b720 0x7f34001e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34000ae370 0x7f34001e1240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 --2- 172.21.15.33:0/2246751460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f340006d3e0 0x7f34001e4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 >> 172.21.15.33:0/2246751460 conn(0x7f340006b030 msgr2=0x7f3400073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 shutdown_connections 2024-03-28T01:06:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.518+0000 7f3405200640 1 -- 172.21.15.33:0/2246751460 wait complete. 2024-03-28T01:06:32.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-03-28T01:06:32.528 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:32.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-03-28T01:06:32.530 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:32.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-03-28T01:06:32.531 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:32.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-03-28T01:06:32.540 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.542+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7b340b9170 con 0x7f7b340b5fe0 2024-03-28T01:06:32.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.542+0000 7f7b2be00640 1 -- 172.21.15.33:0/3948786510 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b2407b670 con 0x7f7b340b5fe0 2024-03-28T01:06:32.545 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7b1406bbd0 msgr2=0x7f7b1406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7b1406bbd0 0x7f7b1406e090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b341416e0 tx=0x7f7b20007e00 comp rx=0 tx=0).stop 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b340b5fe0 msgr2=0x7f7b34148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b340b5fe0 0x7f7b34148c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b24000c00 tx=0x7f7b240046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 shutdown_connections 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7b1406bbd0 0x7f7b1406e090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b340bb7e0 0x7f7b34147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b340b63c0 0x7f7b341472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 --2- 172.21.15.33:0/3948786510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b340b5fe0 0x7f7b34148c10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 >> 172.21.15.33:0/3948786510 conn(0x7f7b34072940 msgr2=0x7f7b340b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 shutdown_connections 2024-03-28T01:06:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.545+0000 7f7b39800640 1 -- 172.21.15.33:0/3948786510 wait complete. 2024-03-28T01:06:32.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-03-28T01:06:32.566 DEBUG:teuthology.parallel:result is None 2024-03-28T01:06:32.566 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-28T01:06:32.566 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:06:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.823+0000 7f9f2f200640 1 Processor -- start 2024-03-28T01:06:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.823+0000 7f9f2f200640 1 -- start start 2024-03-28T01:06:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.824+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f300b8d00 0x7f9f3014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.824+0000 7f9f2f200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f300b55b0 con 0x7f9f300b51d0 2024-03-28T01:06:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.824+0000 7f9f2f200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f300b9190 con 0x7f9f300b6b80 2024-03-28T01:06:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.824+0000 7f9f2f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3014a820 con 0x7f9f300b8d00 2024-03-28T01:06:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.824+0000 7f9f2de00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9f300b51d0 0x7f9f300b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55548/0 (socket says 172.21.15.33:55548) 2024-03-28T01:06:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.824+0000 7f9f2de00640 1 -- 172.21.15.33:0/3725454372 learned_addr learned my addr 172.21.15.33:0/3725454372 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2d400640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f300b8d00 0x7f9f3014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311015078 0 0) 0x7f9f300b55b0 con 0x7f9f300b51d0 2024-03-28T01:06:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f14003520 con 0x7f9f300b51d0 2024-03-28T01:06:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4113913346 0 0) 0x7f9f14003520 con 0x7f9f300b51d0 2024-03-28T01:06:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f300b8d00 msgr2=0x7f9f3014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f300b8d00 0x7f9f3014ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 >> v1:172.21.15.194:6789/0 conn(0x7f9f300b6b80 legacy=0x7f9f3014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300b9800 con 0x7f9f300b51d0 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f9f20001670 con 0x7f9f300b51d0 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 >> v1:172.21.15.33:6789/0 conn(0x7f9f300b51d0 legacy=0x7f9f300b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f140039e0 0x7f9f14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f14004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f14005060 0x7f9f1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1400bf00 con 0x7f9f14004390 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1400c080 con 0x7f9f14005060 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.825+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1400c250 con 0x7f9f140039e0 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2d400640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f14004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2de00640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f140039e0 0x7f9f14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2e800640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f14005060 0x7f9f1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2d400640 1 -- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f140039e0 msgr2=0x7f9f14003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2d400640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f140039e0 0x7f9f14003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2d400640 1 -- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f14005060 msgr2=0x7f9f1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2d400640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f14005060 0x7f9f1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.826+0000 7f9f2d400640 1 -- 172.21.15.33:0/3725454372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14003520 con 0x7f9f14004390 2024-03-28T01:06:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.827+0000 7f9f2d400640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f14004860 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009aa0 tx=0x7f9f14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1df5a6935d6a0fe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.827+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c039070 con 0x7f9f14004390 2024-03-28T01:06:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c02c6b0 con 0x7f9f14004390 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2ca00640 1 -- 172.21.15.33:0/3725454372 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c009670 con 0x7f9f14004390 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2f200640 1 -- 172.21.15.33:0/3725454372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 msgr2=0x7f9f14004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2f200640 1 --2- 172.21.15.33:0/3725454372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f14004860 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009aa0 tx=0x7f9f14005830 comp rx=0 tx=0).stop 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2e800640 1 -- 172.21.15.33:0/3725454372 reap_dead start 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2f200640 1 -- 172.21.15.33:0/3725454372 shutdown_connections 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2f200640 1 -- 172.21.15.33:0/3725454372 >> 172.21.15.33:0/3725454372 conn(0x7f9f30072940 msgr2=0x7f9f30072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2f200640 1 -- 172.21.15.33:0/3725454372 shutdown_connections 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.828+0000 7f9f2f200640 1 -- 172.21.15.33:0/3725454372 wait complete. 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.830+0000 7f9f2f200640 1 Processor -- start 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.830+0000 7f9f2f200640 1 -- start start 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f301498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f301dfe50 0x7f9f300b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f301e02b0 0x7f9f300b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300b39c0 con 0x7f9f14004390 2024-03-28T01:06:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300b44c0 con 0x7f9f301e02b0 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3014ac70 con 0x7f9f301dfe50 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f301498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f301498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40176/0 (socket says 172.21.15.33:40176) 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 -- 172.21.15.33:0/3220110213 learned_addr learned my addr 172.21.15.33:0/3220110213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 -- 172.21.15.33:0/3220110213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f301dfe50 msgr2=0x7f9f300b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f301dfe50 0x7f9f300b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 -- 172.21.15.33:0/3220110213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f301e02b0 msgr2=0x7f9f300b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f301e02b0 0x7f9f300b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.831+0000 7f9f2de00640 1 -- 172.21.15.33:0/3220110213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300aa3c0 con 0x7f9f14004390 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.832+0000 7f9f2de00640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f301498e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000e830 tx=0x7f9f2000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.832+0000 7f9f26000640 1 -- 172.21.15.33:0/3220110213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000c8d0 con 0x7f9f14004390 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.832+0000 7f9f26000640 1 -- 172.21.15.33:0/3220110213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20018070 con 0x7f9f14004390 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.832+0000 7f9f26000640 1 -- 172.21.15.33:0/3220110213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000ccd0 con 0x7f9f14004390 2024-03-28T01:06:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.832+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f30143290 con 0x7f9f14004390 2024-03-28T01:06:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.832+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f300aa0e0 con 0x7f9f14004390 2024-03-28T01:06:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.833+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f300b2aa0 con 0x7f9f14004390 2024-03-28T01:06:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.834+0000 7f9f26000640 1 -- 172.21.15.33:0/3220110213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9f2001d460 con 0x7f9f14004390 2024-03-28T01:06:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.835+0000 7f9f26000640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9f1006bc00 0x7f9f1006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.835+0000 7f9f26000640 1 -- 172.21.15.33:0/3220110213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f2008a910 con 0x7f9f14004390 2024-03-28T01:06:32.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.835+0000 7f9f2d400640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9f1006bc00 0x7f9f1006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:32.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.836+0000 7f9f2d400640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9f1006bc00 0x7f9f1006e0c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c036000 tx=0x7f9f1c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:32.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:32.840+0000 7f9f26000640 1 -- 172.21.15.33:0/3220110213 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9f300b2aa0 con 0x7f9f14004390 2024-03-28T01:06:33.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.032+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f30075310 con 0x7f9f1006bc00 2024-03-28T01:06:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.045+0000 7f9f26000640 1 -- 172.21.15.33:0/3220110213 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f9f30075310 con 0x7f9f1006bc00 2024-03-28T01:06:33.051 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9f1006bc00 msgr2=0x7f9f1006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9f1006bc00 0x7f9f1006e0c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c036000 tx=0x7f9f1c02e000 comp rx=0 tx=0).stop 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 msgr2=0x7f9f301498e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f301498e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000e830 tx=0x7f9f2000ecf0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 shutdown_connections 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9f1006bc00 0x7f9f1006e0c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f301e02b0 0x7f9f300b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f301dfe50 0x7f9f300b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 --2- 172.21.15.33:0/3220110213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f14004390 0x7f9f301498e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 >> 172.21.15.33:0/3220110213 conn(0x7f9f30072940 msgr2=0x7f9f30140e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 shutdown_connections 2024-03-28T01:06:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.051+0000 7f9f2f200640 1 -- 172.21.15.33:0/3220110213 wait complete. 2024-03-28T01:06:33.071 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-28T01:06:32.390188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.693581"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857593+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:06:26.857593+0000","last_peered":"2024-03-28T01:06:26.857593+0000","last_clean":"2024-03-28T01:06:26.857593+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:06:26.857593+0000","last_undegraded":"2024-03-28T01:06:26.857593+0000","last_fullsized":"2024-03-28T01:06:26.857593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711746+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:06:24.711746+0000","last_peered":"2024-03-28T01:06:24.711746+0000","last_clean":"2024-03-28T01:06:24.711746+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:06:24.711746+0000","last_undegraded":"2024-03-28T01:06:24.711746+0000","last_fullsized":"2024-03-28T01:06:24.711746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723513+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:06:23.723513+0000","last_peered":"2024-03-28T01:06:23.723513+0000","last_clean":"2024-03-28T01:06:23.723513+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:06:23.723513+0000","last_undegraded":"2024-03-28T01:06:23.723513+0000","last_fullsized":"2024-03-28T01:06:23.723513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855078+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:06:26.855078+0000","last_peered":"2024-03-28T01:06:26.855078+0000","last_clean":"2024-03-28T01:06:26.855078+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:06:26.855078+0000","last_undegraded":"2024-03-28T01:06:26.855078+0000","last_fullsized":"2024-03-28T01:06:26.855078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856944+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:06:26.856944+0000","last_peered":"2024-03-28T01:06:26.856944+0000","last_clean":"2024-03-28T01:06:26.856944+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:06:26.856944+0000","last_undegraded":"2024-03-28T01:06:26.856944+0000","last_fullsized":"2024-03-28T01:06:26.856944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854320+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:06:26.854320+0000","last_peered":"2024-03-28T01:06:26.854320+0000","last_clean":"2024-03-28T01:06:26.854320+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:06:26.854320+0000","last_undegraded":"2024-03-28T01:06:26.854320+0000","last_fullsized":"2024-03-28T01:06:26.854320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856148+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:06:26.856148+0000","last_peered":"2024-03-28T01:06:26.856148+0000","last_clean":"2024-03-28T01:06:26.856148+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:06:26.856148+0000","last_undegraded":"2024-03-28T01:06:26.856148+0000","last_fullsized":"2024-03-28T01:06:26.856148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723763+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:06:23.723763+0000","last_peered":"2024-03-28T01:06:23.723763+0000","last_clean":"2024-03-28T01:06:23.723763+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:06:23.723763+0000","last_undegraded":"2024-03-28T01:06:23.723763+0000","last_fullsized":"2024-03-28T01:06:23.723763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.858037+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:06:26.858037+0000","last_peered":"2024-03-28T01:06:26.858037+0000","last_clean":"2024-03-28T01:06:26.858037+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:06:26.858037+0000","last_undegraded":"2024-03-28T01:06:26.858037+0000","last_fullsized":"2024-03-28T01:06:26.858037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718027+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:06:24.718027+0000","last_peered":"2024-03-28T01:06:24.718027+0000","last_clean":"2024-03-28T01:06:24.718027+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:06:24.718027+0000","last_undegraded":"2024-03-28T01:06:24.718027+0000","last_fullsized":"2024-03-28T01:06:24.718027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717838+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:06:24.717838+0000","last_peered":"2024-03-28T01:06:24.717838+0000","last_clean":"2024-03-28T01:06:24.717838+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:06:24.717838+0000","last_undegraded":"2024-03-28T01:06:24.717838+0000","last_fullsized":"2024-03-28T01:06:24.717838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727289+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:06:23.727289+0000","last_peered":"2024-03-28T01:06:23.727289+0000","last_clean":"2024-03-28T01:06:23.727289+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:06:23.727289+0000","last_undegraded":"2024-03-28T01:06:23.727289+0000","last_fullsized":"2024-03-28T01:06:23.727289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724224+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:06:23.724224+0000","last_peered":"2024-03-28T01:06:23.724224+0000","last_clean":"2024-03-28T01:06:23.724224+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:06:23.724224+0000","last_undegraded":"2024-03-28T01:06:23.724224+0000","last_fullsized":"2024-03-28T01:06:23.724224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722772+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:06:23.722772+0000","last_peered":"2024-03-28T01:06:23.722772+0000","last_clean":"2024-03-28T01:06:23.722772+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:06:23.722772+0000","last_undegraded":"2024-03-28T01:06:23.722772+0000","last_fullsized":"2024-03-28T01:06:23.722772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710748+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:06:24.710748+0000","last_peered":"2024-03-28T01:06:24.710748+0000","last_clean":"2024-03-28T01:06:24.710748+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:06:24.710748+0000","last_undegraded":"2024-03-28T01:06:24.710748+0000","last_fullsized":"2024-03-28T01:06:24.710748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854169+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:06:26.854169+0000","last_peered":"2024-03-28T01:06:26.854169+0000","last_clean":"2024-03-28T01:06:26.854169+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:06:26.854169+0000","last_undegraded":"2024-03-28T01:06:26.854169+0000","last_fullsized":"2024-03-28T01:06:26.854169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856921+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:06:26.856921+0000","last_peered":"2024-03-28T01:06:26.856921+0000","last_clean":"2024-03-28T01:06:26.856921+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:06:26.856921+0000","last_undegraded":"2024-03-28T01:06:26.856921+0000","last_fullsized":"2024-03-28T01:06:26.856921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261345+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:06:26.261345+0000","last_peered":"2024-03-28T01:06:26.261345+0000","last_clean":"2024-03-28T01:06:26.261345+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:06:26.261345+0000","last_undegraded":"2024-03-28T01:06:26.261345+0000","last_fullsized":"2024-03-28T01:06:26.261345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857651+0000","last_change":"2024-03-28T01:06:22.743819+0000","last_active":"2024-03-28T01:06:26.857651+0000","last_peered":"2024-03-28T01:06:26.857651+0000","last_clean":"2024-03-28T01:06:26.857651+0000","last_became_active":"2024-03-28T01:06:22.742898+0000","last_became_peered":"2024-03-28T01:06:22.742898+0000","last_unstale":"2024-03-28T01:06:26.857651+0000","last_undegraded":"2024-03-28T01:06:26.857651+0000","last_fullsized":"2024-03-28T01:06:26.857651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717295+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:06:24.717295+0000","last_peered":"2024-03-28T01:06:24.717295+0000","last_clean":"2024-03-28T01:06:24.717295+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:06:24.717295+0000","last_undegraded":"2024-03-28T01:06:24.717295+0000","last_fullsized":"2024-03-28T01:06:24.717295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722929+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:06:23.722929+0000","last_peered":"2024-03-28T01:06:23.722929+0000","last_clean":"2024-03-28T01:06:23.722929+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:06:23.722929+0000","last_undegraded":"2024-03-28T01:06:23.722929+0000","last_fullsized":"2024-03-28T01:06:23.722929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854322+0000","last_change":"2024-03-28T01:06:22.749132+0000","last_active":"2024-03-28T01:06:26.854322+0000","last_peered":"2024-03-28T01:06:26.854322+0000","last_clean":"2024-03-28T01:06:26.854322+0000","last_became_active":"2024-03-28T01:06:22.748012+0000","last_became_peered":"2024-03-28T01:06:22.748012+0000","last_unstale":"2024-03-28T01:06:26.854322+0000","last_undegraded":"2024-03-28T01:06:26.854322+0000","last_fullsized":"2024-03-28T01:06:26.854322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724138+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:06:23.724138+0000","last_peered":"2024-03-28T01:06:23.724138+0000","last_clean":"2024-03-28T01:06:23.724138+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:06:23.724138+0000","last_undegraded":"2024-03-28T01:06:23.724138+0000","last_fullsized":"2024-03-28T01:06:23.724138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854193+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:06:26.854193+0000","last_peered":"2024-03-28T01:06:26.854193+0000","last_clean":"2024-03-28T01:06:26.854193+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:06:26.854193+0000","last_undegraded":"2024-03-28T01:06:26.854193+0000","last_fullsized":"2024-03-28T01:06:26.854193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724983+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:06:23.724983+0000","last_peered":"2024-03-28T01:06:23.724983+0000","last_clean":"2024-03-28T01:06:23.724983+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:06:23.724983+0000","last_undegraded":"2024-03-28T01:06:23.724983+0000","last_fullsized":"2024-03-28T01:06:23.724983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718057+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:06:24.718057+0000","last_peered":"2024-03-28T01:06:24.718057+0000","last_clean":"2024-03-28T01:06:24.718057+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:06:24.718057+0000","last_undegraded":"2024-03-28T01:06:24.718057+0000","last_fullsized":"2024-03-28T01:06:24.718057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711534+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:06:24.711534+0000","last_peered":"2024-03-28T01:06:24.711534+0000","last_clean":"2024-03-28T01:06:24.711534+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:06:24.711534+0000","last_undegraded":"2024-03-28T01:06:24.711534+0000","last_fullsized":"2024-03-28T01:06:24.711534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725154+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:06:23.725154+0000","last_peered":"2024-03-28T01:06:23.725154+0000","last_clean":"2024-03-28T01:06:23.725154+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:06:23.725154+0000","last_undegraded":"2024-03-28T01:06:23.725154+0000","last_fullsized":"2024-03-28T01:06:23.725154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723116+0000","last_change":"2024-03-28T01:06:22.731272+0000","last_active":"2024-03-28T01:06:23.723116+0000","last_peered":"2024-03-28T01:06:23.723116+0000","last_clean":"2024-03-28T01:06:23.723116+0000","last_became_active":"2024-03-28T01:06:22.730261+0000","last_became_peered":"2024-03-28T01:06:22.730261+0000","last_unstale":"2024-03-28T01:06:23.723116+0000","last_undegraded":"2024-03-28T01:06:23.723116+0000","last_fullsized":"2024-03-28T01:06:23.723116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855545+0000","last_change":"2024-03-28T01:06:22.746590+0000","last_active":"2024-03-28T01:06:26.855545+0000","last_peered":"2024-03-28T01:06:26.855545+0000","last_clean":"2024-03-28T01:06:26.855545+0000","last_became_active":"2024-03-28T01:06:22.746299+0000","last_became_peered":"2024-03-28T01:06:22.746299+0000","last_unstale":"2024-03-28T01:06:26.855545+0000","last_undegraded":"2024-03-28T01:06:26.855545+0000","last_fullsized":"2024-03-28T01:06:26.855545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261649+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:06:26.261649+0000","last_peered":"2024-03-28T01:06:26.261649+0000","last_clean":"2024-03-28T01:06:26.261649+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:06:26.261649+0000","last_undegraded":"2024-03-28T01:06:26.261649+0000","last_fullsized":"2024-03-28T01:06:26.261649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717294+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:06:24.717294+0000","last_peered":"2024-03-28T01:06:24.717294+0000","last_clean":"2024-03-28T01:06:24.717294+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:06:24.717294+0000","last_undegraded":"2024-03-28T01:06:24.717294+0000","last_fullsized":"2024-03-28T01:06:24.717294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726504+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:06:23.726504+0000","last_peered":"2024-03-28T01:06:23.726504+0000","last_clean":"2024-03-28T01:06:23.726504+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:06:23.726504+0000","last_undegraded":"2024-03-28T01:06:23.726504+0000","last_fullsized":"2024-03-28T01:06:23.726504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724518+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:06:23.724518+0000","last_peered":"2024-03-28T01:06:23.724518+0000","last_clean":"2024-03-28T01:06:23.724518+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:06:23.724518+0000","last_undegraded":"2024-03-28T01:06:23.724518+0000","last_fullsized":"2024-03-28T01:06:23.724518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723265+0000","last_change":"2024-03-28T01:06:22.717986+0000","last_active":"2024-03-28T01:06:23.723265+0000","last_peered":"2024-03-28T01:06:23.723265+0000","last_clean":"2024-03-28T01:06:23.723265+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:06:23.723265+0000","last_undegraded":"2024-03-28T01:06:23.723265+0000","last_fullsized":"2024-03-28T01:06:23.723265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717592+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:06:24.717592+0000","last_peered":"2024-03-28T01:06:24.717592+0000","last_clean":"2024-03-28T01:06:24.717592+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:06:24.717592+0000","last_undegraded":"2024-03-28T01:06:24.717592+0000","last_fullsized":"2024-03-28T01:06:24.717592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724278+0000","last_change":"2024-03-28T01:06:21.721278+0000","last_active":"2024-03-28T01:06:23.724278+0000","last_peered":"2024-03-28T01:06:23.724278+0000","last_clean":"2024-03-28T01:06:23.724278+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:06:23.724278+0000","last_undegraded":"2024-03-28T01:06:23.724278+0000","last_fullsized":"2024-03-28T01:06:23.724278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723815+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:06:23.723815+0000","last_peered":"2024-03-28T01:06:23.723815+0000","last_clean":"2024-03-28T01:06:23.723815+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:06:23.723815+0000","last_undegraded":"2024-03-28T01:06:23.723815+0000","last_fullsized":"2024-03-28T01:06:23.723815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854498+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:06:26.854498+0000","last_peered":"2024-03-28T01:06:26.854498+0000","last_clean":"2024-03-28T01:06:26.854498+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:06:26.854498+0000","last_undegraded":"2024-03-28T01:06:26.854498+0000","last_fullsized":"2024-03-28T01:06:26.854498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710729+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:06:24.710729+0000","last_peered":"2024-03-28T01:06:24.710729+0000","last_clean":"2024-03-28T01:06:24.710729+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:06:24.710729+0000","last_undegraded":"2024-03-28T01:06:24.710729+0000","last_fullsized":"2024-03-28T01:06:24.710729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717704+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:06:24.717704+0000","last_peered":"2024-03-28T01:06:24.717704+0000","last_clean":"2024-03-28T01:06:24.717704+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:06:24.717704+0000","last_undegraded":"2024-03-28T01:06:24.717704+0000","last_fullsized":"2024-03-28T01:06:24.717704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855761+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:06:26.855761+0000","last_peered":"2024-03-28T01:06:26.855761+0000","last_clean":"2024-03-28T01:06:26.855761+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:06:26.855761+0000","last_undegraded":"2024-03-28T01:06:26.855761+0000","last_fullsized":"2024-03-28T01:06:26.855761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855830+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:06:26.855830+0000","last_peered":"2024-03-28T01:06:26.855830+0000","last_clean":"2024-03-28T01:06:26.855830+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:06:26.855830+0000","last_undegraded":"2024-03-28T01:06:26.855830+0000","last_fullsized":"2024-03-28T01:06:26.855830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724058+0000","last_change":"2024-03-28T01:06:22.743892+0000","last_active":"2024-03-28T01:06:23.724058+0000","last_peered":"2024-03-28T01:06:23.724058+0000","last_clean":"2024-03-28T01:06:23.724058+0000","last_became_active":"2024-03-28T01:06:22.743113+0000","last_became_peered":"2024-03-28T01:06:22.743113+0000","last_unstale":"2024-03-28T01:06:23.724058+0000","last_undegraded":"2024-03-28T01:06:23.724058+0000","last_fullsized":"2024-03-28T01:06:23.724058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711550+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:06:24.711550+0000","last_peered":"2024-03-28T01:06:24.711550+0000","last_clean":"2024-03-28T01:06:24.711550+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:06:24.711550+0000","last_undegraded":"2024-03-28T01:06:24.711550+0000","last_fullsized":"2024-03-28T01:06:24.711550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727568+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:06:23.727568+0000","last_peered":"2024-03-28T01:06:23.727568+0000","last_clean":"2024-03-28T01:06:23.727568+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:06:23.727568+0000","last_undegraded":"2024-03-28T01:06:23.727568+0000","last_fullsized":"2024-03-28T01:06:23.727568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717385+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:06:24.717385+0000","last_peered":"2024-03-28T01:06:24.717385+0000","last_clean":"2024-03-28T01:06:24.717385+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:06:24.717385+0000","last_undegraded":"2024-03-28T01:06:24.717385+0000","last_fullsized":"2024-03-28T01:06:24.717385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856677+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:06:26.856677+0000","last_peered":"2024-03-28T01:06:26.856677+0000","last_clean":"2024-03-28T01:06:26.856677+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:06:26.856677+0000","last_undegraded":"2024-03-28T01:06:26.856677+0000","last_fullsized":"2024-03-28T01:06:26.856677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723346+0000","last_change":"2024-03-28T01:06:21.727305+0000","last_active":"2024-03-28T01:06:23.723346+0000","last_peered":"2024-03-28T01:06:23.723346+0000","last_clean":"2024-03-28T01:06:23.723346+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:06:23.723346+0000","last_undegraded":"2024-03-28T01:06:23.723346+0000","last_fullsized":"2024-03-28T01:06:23.723346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722951+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:06:23.722951+0000","last_peered":"2024-03-28T01:06:23.722951+0000","last_clean":"2024-03-28T01:06:23.722951+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:06:23.722951+0000","last_undegraded":"2024-03-28T01:06:23.722951+0000","last_fullsized":"2024-03-28T01:06:23.722951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854839+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:06:26.854839+0000","last_peered":"2024-03-28T01:06:26.854839+0000","last_clean":"2024-03-28T01:06:26.854839+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:06:26.854839+0000","last_undegraded":"2024-03-28T01:06:26.854839+0000","last_fullsized":"2024-03-28T01:06:26.854839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857353+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:06:26.857353+0000","last_peered":"2024-03-28T01:06:26.857353+0000","last_clean":"2024-03-28T01:06:26.857353+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:06:26.857353+0000","last_undegraded":"2024-03-28T01:06:26.857353+0000","last_fullsized":"2024-03-28T01:06:26.857353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717022+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:06:24.717022+0000","last_peered":"2024-03-28T01:06:24.717022+0000","last_clean":"2024-03-28T01:06:24.717022+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:06:24.717022+0000","last_undegraded":"2024-03-28T01:06:24.717022+0000","last_fullsized":"2024-03-28T01:06:24.717022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717552+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:06:24.717552+0000","last_peered":"2024-03-28T01:06:24.717552+0000","last_clean":"2024-03-28T01:06:24.717552+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:06:24.717552+0000","last_undegraded":"2024-03-28T01:06:24.717552+0000","last_fullsized":"2024-03-28T01:06:24.717552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261265+0000","last_change":"2024-03-28T01:06:21.719466+0000","last_active":"2024-03-28T01:06:26.261265+0000","last_peered":"2024-03-28T01:06:26.261265+0000","last_clean":"2024-03-28T01:06:26.261265+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:06:26.261265+0000","last_undegraded":"2024-03-28T01:06:26.261265+0000","last_fullsized":"2024-03-28T01:06:26.261265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856277+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:06:26.856277+0000","last_peered":"2024-03-28T01:06:26.856277+0000","last_clean":"2024-03-28T01:06:26.856277+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:06:26.856277+0000","last_undegraded":"2024-03-28T01:06:26.856277+0000","last_fullsized":"2024-03-28T01:06:26.856277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727523+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:06:23.727523+0000","last_peered":"2024-03-28T01:06:23.727523+0000","last_clean":"2024-03-28T01:06:23.727523+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:06:23.727523+0000","last_undegraded":"2024-03-28T01:06:23.727523+0000","last_fullsized":"2024-03-28T01:06:23.727523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856873+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:06:26.856873+0000","last_peered":"2024-03-28T01:06:26.856873+0000","last_clean":"2024-03-28T01:06:26.856873+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:06:26.856873+0000","last_undegraded":"2024-03-28T01:06:26.856873+0000","last_fullsized":"2024-03-28T01:06:26.856873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261460+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:06:26.261460+0000","last_peered":"2024-03-28T01:06:26.261460+0000","last_clean":"2024-03-28T01:06:26.261460+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:06:26.261460+0000","last_undegraded":"2024-03-28T01:06:26.261460+0000","last_fullsized":"2024-03-28T01:06:26.261460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.858459+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:06:26.858459+0000","last_peered":"2024-03-28T01:06:26.858459+0000","last_clean":"2024-03-28T01:06:26.858459+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:06:26.858459+0000","last_undegraded":"2024-03-28T01:06:26.858459+0000","last_fullsized":"2024-03-28T01:06:26.858459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711878+0000","last_change":"2024-03-28T01:06:21.718159+0000","last_active":"2024-03-28T01:06:24.711878+0000","last_peered":"2024-03-28T01:06:24.711878+0000","last_clean":"2024-03-28T01:06:24.711878+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:06:24.711878+0000","last_undegraded":"2024-03-28T01:06:24.711878+0000","last_fullsized":"2024-03-28T01:06:24.711878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855554+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:06:26.855554+0000","last_peered":"2024-03-28T01:06:26.855554+0000","last_clean":"2024-03-28T01:06:26.855554+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:06:26.855554+0000","last_undegraded":"2024-03-28T01:06:26.855554+0000","last_fullsized":"2024-03-28T01:06:26.855554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261512+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:06:26.261512+0000","last_peered":"2024-03-28T01:06:26.261512+0000","last_clean":"2024-03-28T01:06:26.261512+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:06:26.261512+0000","last_undegraded":"2024-03-28T01:06:26.261512+0000","last_fullsized":"2024-03-28T01:06:26.261512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856940+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:06:26.856940+0000","last_peered":"2024-03-28T01:06:26.856940+0000","last_clean":"2024-03-28T01:06:26.856940+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:06:26.856940+0000","last_undegraded":"2024-03-28T01:06:26.856940+0000","last_fullsized":"2024-03-28T01:06:26.856940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856017+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:06:26.856017+0000","last_peered":"2024-03-28T01:06:26.856017+0000","last_clean":"2024-03-28T01:06:26.856017+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:06:26.856017+0000","last_undegraded":"2024-03-28T01:06:26.856017+0000","last_fullsized":"2024-03-28T01:06:26.856017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854326+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:06:26.854326+0000","last_peered":"2024-03-28T01:06:26.854326+0000","last_clean":"2024-03-28T01:06:26.854326+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:06:26.854326+0000","last_undegraded":"2024-03-28T01:06:26.854326+0000","last_fullsized":"2024-03-28T01:06:26.854326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724610+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:06:23.724610+0000","last_peered":"2024-03-28T01:06:23.724610+0000","last_clean":"2024-03-28T01:06:23.724610+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:06:23.724610+0000","last_undegraded":"2024-03-28T01:06:23.724610+0000","last_fullsized":"2024-03-28T01:06:23.724610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857175+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:06:26.857175+0000","last_peered":"2024-03-28T01:06:26.857175+0000","last_clean":"2024-03-28T01:06:26.857175+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:06:26.857175+0000","last_undegraded":"2024-03-28T01:06:26.857175+0000","last_fullsized":"2024-03-28T01:06:26.857175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722723+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:06:23.722723+0000","last_peered":"2024-03-28T01:06:23.722723+0000","last_clean":"2024-03-28T01:06:23.722723+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:06:23.722723+0000","last_undegraded":"2024-03-28T01:06:23.722723+0000","last_fullsized":"2024-03-28T01:06:23.722723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261507+0000","last_change":"2024-03-28T01:06:21.720753+0000","last_active":"2024-03-28T01:06:26.261507+0000","last_peered":"2024-03-28T01:06:26.261507+0000","last_clean":"2024-03-28T01:06:26.261507+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:06:26.261507+0000","last_undegraded":"2024-03-28T01:06:26.261507+0000","last_fullsized":"2024-03-28T01:06:26.261507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710942+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:06:24.710942+0000","last_peered":"2024-03-28T01:06:24.710942+0000","last_clean":"2024-03-28T01:06:24.710942+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:06:24.710942+0000","last_undegraded":"2024-03-28T01:06:24.710942+0000","last_fullsized":"2024-03-28T01:06:24.710942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855089+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:06:26.855089+0000","last_peered":"2024-03-28T01:06:26.855089+0000","last_clean":"2024-03-28T01:06:26.855089+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:06:26.855089+0000","last_undegraded":"2024-03-28T01:06:26.855089+0000","last_fullsized":"2024-03-28T01:06:26.855089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727032+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:06:23.727032+0000","last_peered":"2024-03-28T01:06:23.727032+0000","last_clean":"2024-03-28T01:06:23.727032+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:06:23.727032+0000","last_undegraded":"2024-03-28T01:06:23.727032+0000","last_fullsized":"2024-03-28T01:06:23.727032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711510+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:06:24.711510+0000","last_peered":"2024-03-28T01:06:24.711510+0000","last_clean":"2024-03-28T01:06:24.711510+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:06:24.711510+0000","last_undegraded":"2024-03-28T01:06:24.711510+0000","last_fullsized":"2024-03-28T01:06:24.711510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855665+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:06:26.855665+0000","last_peered":"2024-03-28T01:06:26.855665+0000","last_clean":"2024-03-28T01:06:26.855665+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:06:26.855665+0000","last_undegraded":"2024-03-28T01:06:26.855665+0000","last_fullsized":"2024-03-28T01:06:26.855665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857875+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:06:26.857875+0000","last_peered":"2024-03-28T01:06:26.857875+0000","last_clean":"2024-03-28T01:06:26.857875+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:06:26.857875+0000","last_undegraded":"2024-03-28T01:06:26.857875+0000","last_fullsized":"2024-03-28T01:06:26.857875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711037+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:06:24.711037+0000","last_peered":"2024-03-28T01:06:24.711037+0000","last_clean":"2024-03-28T01:06:24.711037+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:06:24.711037+0000","last_undegraded":"2024-03-28T01:06:24.711037+0000","last_fullsized":"2024-03-28T01:06:24.711037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.715740+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:06:24.715740+0000","last_peered":"2024-03-28T01:06:24.715740+0000","last_clean":"2024-03-28T01:06:24.715740+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:06:24.715740+0000","last_undegraded":"2024-03-28T01:06:24.715740+0000","last_fullsized":"2024-03-28T01:06:24.715740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:34.313143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725201+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:06:23.725201+0000","last_peered":"2024-03-28T01:06:23.725201+0000","last_clean":"2024-03-28T01:06:23.725201+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:06:23.725201+0000","last_undegraded":"2024-03-28T01:06:23.725201+0000","last_fullsized":"2024-03-28T01:06:23.725201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717736+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:06:24.717736+0000","last_peered":"2024-03-28T01:06:24.717736+0000","last_clean":"2024-03-28T01:06:24.717736+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:06:24.717736+0000","last_undegraded":"2024-03-28T01:06:24.717736+0000","last_fullsized":"2024-03-28T01:06:24.717736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723864+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:06:23.723864+0000","last_peered":"2024-03-28T01:06:23.723864+0000","last_clean":"2024-03-28T01:06:23.723864+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:06:23.723864+0000","last_undegraded":"2024-03-28T01:06:23.723864+0000","last_fullsized":"2024-03-28T01:06:23.723864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855061+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:06:26.855061+0000","last_peered":"2024-03-28T01:06:26.855061+0000","last_clean":"2024-03-28T01:06:26.855061+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:06:26.855061+0000","last_undegraded":"2024-03-28T01:06:26.855061+0000","last_fullsized":"2024-03-28T01:06:26.855061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726343+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:06:23.726343+0000","last_peered":"2024-03-28T01:06:23.726343+0000","last_clean":"2024-03-28T01:06:23.726343+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:06:23.726343+0000","last_undegraded":"2024-03-28T01:06:23.726343+0000","last_fullsized":"2024-03-28T01:06:23.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710944+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:06:24.710944+0000","last_peered":"2024-03-28T01:06:24.710944+0000","last_clean":"2024-03-28T01:06:24.710944+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:06:24.710944+0000","last_undegraded":"2024-03-28T01:06:24.710944+0000","last_fullsized":"2024-03-28T01:06:24.710944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723935+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:06:23.723935+0000","last_peered":"2024-03-28T01:06:23.723935+0000","last_clean":"2024-03-28T01:06:23.723935+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:06:23.723935+0000","last_undegraded":"2024-03-28T01:06:23.723935+0000","last_fullsized":"2024-03-28T01:06:23.723935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711041+0000","last_change":"2024-03-28T01:06:21.715183+0000","last_active":"2024-03-28T01:06:24.711041+0000","last_peered":"2024-03-28T01:06:24.711041+0000","last_clean":"2024-03-28T01:06:24.711041+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:06:24.711041+0000","last_undegraded":"2024-03-28T01:06:24.711041+0000","last_fullsized":"2024-03-28T01:06:24.711041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724403+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:06:23.724403+0000","last_peered":"2024-03-28T01:06:23.724403+0000","last_clean":"2024-03-28T01:06:23.724403+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:06:23.724403+0000","last_undegraded":"2024-03-28T01:06:23.724403+0000","last_fullsized":"2024-03-28T01:06:23.724403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857617+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:06:26.857617+0000","last_peered":"2024-03-28T01:06:26.857617+0000","last_clean":"2024-03-28T01:06:26.857617+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:06:26.857617+0000","last_undegraded":"2024-03-28T01:06:26.857617+0000","last_fullsized":"2024-03-28T01:06:26.857617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717267+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:06:24.717267+0000","last_peered":"2024-03-28T01:06:24.717267+0000","last_clean":"2024-03-28T01:06:24.717267+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:06:24.717267+0000","last_undegraded":"2024-03-28T01:06:24.717267+0000","last_fullsized":"2024-03-28T01:06:24.717267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855244+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:06:26.855244+0000","last_peered":"2024-03-28T01:06:26.855244+0000","last_clean":"2024-03-28T01:06:26.855244+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:06:26.855244+0000","last_undegraded":"2024-03-28T01:06:26.855244+0000","last_fullsized":"2024-03-28T01:06:26.855244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857157+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:06:26.857157+0000","last_peered":"2024-03-28T01:06:26.857157+0000","last_clean":"2024-03-28T01:06:26.857157+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:06:26.857157+0000","last_undegraded":"2024-03-28T01:06:26.857157+0000","last_fullsized":"2024-03-28T01:06:26.857157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855081+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:06:26.855081+0000","last_peered":"2024-03-28T01:06:26.855081+0000","last_clean":"2024-03-28T01:06:26.855081+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:06:26.855081+0000","last_undegraded":"2024-03-28T01:06:26.855081+0000","last_fullsized":"2024-03-28T01:06:26.855081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857206+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:06:26.857206+0000","last_peered":"2024-03-28T01:06:26.857206+0000","last_clean":"2024-03-28T01:06:26.857206+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:06:26.857206+0000","last_undegraded":"2024-03-28T01:06:26.857206+0000","last_fullsized":"2024-03-28T01:06:26.857206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723980+0000","last_change":"2024-03-28T01:06:22.742813+0000","last_active":"2024-03-28T01:06:23.723980+0000","last_peered":"2024-03-28T01:06:23.723980+0000","last_clean":"2024-03-28T01:06:23.723980+0000","last_became_active":"2024-03-28T01:06:22.736152+0000","last_became_peered":"2024-03-28T01:06:22.736152+0000","last_unstale":"2024-03-28T01:06:23.723980+0000","last_undegraded":"2024-03-28T01:06:23.723980+0000","last_fullsized":"2024-03-28T01:06:23.723980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725147+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:06:23.725147+0000","last_peered":"2024-03-28T01:06:23.725147+0000","last_clean":"2024-03-28T01:06:23.725147+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:06:23.725147+0000","last_undegraded":"2024-03-28T01:06:23.725147+0000","last_fullsized":"2024-03-28T01:06:23.725147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:32.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725002+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:06:23.725002+0000","last_peered":"2024-03-28T01:06:23.725002+0000","last_clean":"2024-03-28T01:06:23.725002+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:06:23.725002+0000","last_undegraded":"2024-03-28T01:06:23.725002+0000","last_fullsized":"2024-03-28T01:06:23.725002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724107+0000","last_change":"2024-03-28T01:06:21.727858+0000","last_active":"2024-03-28T01:06:23.724107+0000","last_peered":"2024-03-28T01:06:23.724107+0000","last_clean":"2024-03-28T01:06:23.724107+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:06:23.724107+0000","last_undegraded":"2024-03-28T01:06:23.724107+0000","last_fullsized":"2024-03-28T01:06:23.724107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:15:55.049218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:06:33.075 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:06:33.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.322+0000 7f67e9600640 1 Processor -- start 2024-03-28T01:06:33.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.322+0000 7f67e9600640 1 -- start start 2024-03-28T01:06:33.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.324+0000 7f67e9600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 0x7f67e4150860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e9600640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40b2510 con 0x7f67e413f500 2024-03-28T01:06:33.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e9600640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40b4670 con 0x7f67e40b2130 2024-03-28T01:06:33.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e9600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e406bef0 con 0x7f67e4148cf0 2024-03-28T01:06:33.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e3400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f67e413f500 0x7f67e41485e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55558/0 (socket says 172.21.15.33:55558) 2024-03-28T01:06:33.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e3400640 1 -- 172.21.15.33:0/3786080382 learned_addr learned my addr 172.21.15.33:0/3786080382 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e8c00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 0x7f67e4150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153038417 0 0) 0x7f67e40b2510 con 0x7f67e413f500 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.325+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c8003520 con 0x7f67e413f500 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496147686 0 0) 0x7f67c8003520 con 0x7f67e413f500 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 msgr2=0x7f67e4150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 0x7f67e4150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 >> v1:172.21.15.194:6789/0 conn(0x7f67e40b2130 legacy=0x7f67e413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40b4c40 con 0x7f67e413f500 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f67d0001670 con 0x7f67e413f500 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 >> v1:172.21.15.33:6789/0 conn(0x7f67e413f500 legacy=0x7f67e41485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c80039e0 0x7f67c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 0x7f67c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e406bef0 con 0x7f67c8004d40 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c800bb50 con 0x7f67c80039e0 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c800bd50 con 0x7f67e4148cf0 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e8c00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.326+0000 7f67e3e00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c80039e0 0x7f67c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.327+0000 7f67e3400640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 0x7f67c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.327+0000 7f67e8c00640 1 -- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 msgr2=0x7f67c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.327+0000 7f67e8c00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 0x7f67c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.327+0000 7f67e8c00640 1 -- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c80039e0 msgr2=0x7f67c8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.327+0000 7f67e8c00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c80039e0 0x7f67c8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.327+0000 7f67e8c00640 1 -- 172.21.15.33:0/3786080382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8003520 con 0x7f67c8004d40 2024-03-28T01:06:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.327+0000 7f67e3400640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4148cf0 0x7f67c8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.328+0000 7f67e8c00640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009f90 tx=0x7f67c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72838e19ebc01f39 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.328+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc039070 con 0x7f67c8004d40 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.328+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc02cbd0 con 0x7f67c8004d40 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.328+0000 7f67e2a00640 1 -- 172.21.15.33:0/3786080382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc009c10 con 0x7f67c8004d40 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.328+0000 7f67e9600640 1 -- 172.21.15.33:0/3786080382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8004d40 msgr2=0x7f67c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.328+0000 7f67e9600640 1 --2- 172.21.15.33:0/3786080382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009f90 tx=0x7f67c8005830 comp rx=0 tx=0).stop 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.328+0000 7f67e8c00640 1 -- 172.21.15.33:0/3786080382 reap_dead start 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.329+0000 7f67e9600640 1 -- 172.21.15.33:0/3786080382 shutdown_connections 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.329+0000 7f67e9600640 1 -- 172.21.15.33:0/3786080382 >> 172.21.15.33:0/3786080382 conn(0x7f67e40b89c0 msgr2=0x7f67e40b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.329+0000 7f67e9600640 1 -- 172.21.15.33:0/3786080382 shutdown_connections 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.329+0000 7f67e9600640 1 -- 172.21.15.33:0/3786080382 wait complete. 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.330+0000 7f67e9600640 1 Processor -- start 2024-03-28T01:06:33.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.330+0000 7f67e9600640 1 -- start start 2024-03-28T01:06:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e9600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40aeb90 0x7f67e41390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e9600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4139700 0x7f67e40b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e9600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4137850 0x7f67e4137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e9600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4072d20 con 0x7f67e40aeb90 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e9600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40b4ec0 con 0x7f67e4139700 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e9600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40aef70 con 0x7f67e4137850 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40aeb90 0x7f67e41390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40aeb90 0x7f67e41390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40208/0 (socket says 172.21.15.33:40208) 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.331+0000 7f67e3e00640 1 -- 172.21.15.33:0/1354178532 learned_addr learned my addr 172.21.15.33:0/1354178532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3400640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4139700 0x7f67e40b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e8c00640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4137850 0x7f67e4137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3e00640 1 -- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4137850 msgr2=0x7f67e4137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3e00640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4137850 0x7f67e4137ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3e00640 1 -- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4139700 msgr2=0x7f67e40b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3e00640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4139700 0x7f67e40b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3e00640 1 -- 172.21.15.33:0/1354178532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4139e70 con 0x7f67e40aeb90 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3400640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4139700 0x7f67e40b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:33.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e8c00640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4137850 0x7f67e4137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67e3e00640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40aeb90 0x7f67e41390d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e800 tx=0x7f67d400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67dbe00640 1 -- 172.21.15.33:0/1354178532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d40050d0 con 0x7f67e40aeb90 2024-03-28T01:06:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67dbe00640 1 -- 172.21.15.33:0/1354178532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d4018070 con 0x7f67e40aeb90 2024-03-28T01:06:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.332+0000 7f67dbe00640 1 -- 172.21.15.33:0/1354178532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400c6c0 con 0x7f67e40aeb90 2024-03-28T01:06:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.333+0000 7f67e9600640 1 -- 172.21.15.33:0/1354178532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e4139b90 con 0x7f67e40aeb90 2024-03-28T01:06:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.333+0000 7f67e9600640 1 -- 172.21.15.33:0/1354178532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e4138220 con 0x7f67e40aeb90 2024-03-28T01:06:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.334+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e4075310 con 0x7f67e40aeb90 2024-03-28T01:06:33.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.336+0000 7f67dbe00640 1 -- 172.21.15.33:0/1354178532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f67d400c860 con 0x7f67e40aeb90 2024-03-28T01:06:33.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.337+0000 7f67dbe00640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f67c406bc00 0x7f67c406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.337+0000 7f67dbe00640 1 -- 172.21.15.33:0/1354178532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67d408a940 con 0x7f67e40aeb90 2024-03-28T01:06:33.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.337+0000 7f67e3400640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f67c406bc00 0x7f67c406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.337+0000 7f67e3400640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f67c406bc00 0x7f67c406e0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f67e413f1e0 tx=0x7f67d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.341+0000 7f67dbe00640 1 -- 172.21.15.33:0/1354178532 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f67d4059390 con 0x7f67e40aeb90 2024-03-28T01:06:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.531+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67e4138cf0 con 0x7f67c406bc00 2024-03-28T01:06:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.553+0000 7f67dbe00640 1 -- 172.21.15.33:0/1354178532 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f67e4138cf0 con 0x7f67c406bc00 2024-03-28T01:06:33.557 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:33.560 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.558+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f67c406bc00 msgr2=0x7f67c406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.558+0000 7f67d9600640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f67c406bc00 0x7f67c406e0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f67e413f1e0 tx=0x7f67d0009040 comp rx=0 tx=0).stop 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.558+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40aeb90 msgr2=0x7f67e41390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.558+0000 7f67d9600640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40aeb90 0x7f67e41390d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e800 tx=0x7f67d400ecc0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 shutdown_connections 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f67c406bc00 0x7f67c406e0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67e4137850 0x7f67e4137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4139700 0x7f67e40b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 --2- 172.21.15.33:0/1354178532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40aeb90 0x7f67e41390d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 >> 172.21.15.33:0/1354178532 conn(0x7f67e40b89c0 msgr2=0x7f67e413a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:33.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 shutdown_connections 2024-03-28T01:06:33.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.559+0000 7f67d9600640 1 -- 172.21.15.33:0/1354178532 wait complete. 2024-03-28T01:06:33.579 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-28T01:06:32.390188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.693581"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857593+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:06:26.857593+0000","last_peered":"2024-03-28T01:06:26.857593+0000","last_clean":"2024-03-28T01:06:26.857593+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:06:26.857593+0000","last_undegraded":"2024-03-28T01:06:26.857593+0000","last_fullsized":"2024-03-28T01:06:26.857593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711746+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:06:24.711746+0000","last_peered":"2024-03-28T01:06:24.711746+0000","last_clean":"2024-03-28T01:06:24.711746+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:06:24.711746+0000","last_undegraded":"2024-03-28T01:06:24.711746+0000","last_fullsized":"2024-03-28T01:06:24.711746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723513+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:06:23.723513+0000","last_peered":"2024-03-28T01:06:23.723513+0000","last_clean":"2024-03-28T01:06:23.723513+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:06:23.723513+0000","last_undegraded":"2024-03-28T01:06:23.723513+0000","last_fullsized":"2024-03-28T01:06:23.723513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855078+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:06:26.855078+0000","last_peered":"2024-03-28T01:06:26.855078+0000","last_clean":"2024-03-28T01:06:26.855078+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:06:26.855078+0000","last_undegraded":"2024-03-28T01:06:26.855078+0000","last_fullsized":"2024-03-28T01:06:26.855078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856944+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:06:26.856944+0000","last_peered":"2024-03-28T01:06:26.856944+0000","last_clean":"2024-03-28T01:06:26.856944+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:06:26.856944+0000","last_undegraded":"2024-03-28T01:06:26.856944+0000","last_fullsized":"2024-03-28T01:06:26.856944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854320+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:06:26.854320+0000","last_peered":"2024-03-28T01:06:26.854320+0000","last_clean":"2024-03-28T01:06:26.854320+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:06:26.854320+0000","last_undegraded":"2024-03-28T01:06:26.854320+0000","last_fullsized":"2024-03-28T01:06:26.854320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856148+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:06:26.856148+0000","last_peered":"2024-03-28T01:06:26.856148+0000","last_clean":"2024-03-28T01:06:26.856148+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:06:26.856148+0000","last_undegraded":"2024-03-28T01:06:26.856148+0000","last_fullsized":"2024-03-28T01:06:26.856148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723763+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:06:23.723763+0000","last_peered":"2024-03-28T01:06:23.723763+0000","last_clean":"2024-03-28T01:06:23.723763+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:06:23.723763+0000","last_undegraded":"2024-03-28T01:06:23.723763+0000","last_fullsized":"2024-03-28T01:06:23.723763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.858037+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:06:26.858037+0000","last_peered":"2024-03-28T01:06:26.858037+0000","last_clean":"2024-03-28T01:06:26.858037+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:06:26.858037+0000","last_undegraded":"2024-03-28T01:06:26.858037+0000","last_fullsized":"2024-03-28T01:06:26.858037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718027+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:06:24.718027+0000","last_peered":"2024-03-28T01:06:24.718027+0000","last_clean":"2024-03-28T01:06:24.718027+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:06:24.718027+0000","last_undegraded":"2024-03-28T01:06:24.718027+0000","last_fullsized":"2024-03-28T01:06:24.718027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717838+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:06:24.717838+0000","last_peered":"2024-03-28T01:06:24.717838+0000","last_clean":"2024-03-28T01:06:24.717838+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:06:24.717838+0000","last_undegraded":"2024-03-28T01:06:24.717838+0000","last_fullsized":"2024-03-28T01:06:24.717838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727289+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:06:23.727289+0000","last_peered":"2024-03-28T01:06:23.727289+0000","last_clean":"2024-03-28T01:06:23.727289+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:06:23.727289+0000","last_undegraded":"2024-03-28T01:06:23.727289+0000","last_fullsized":"2024-03-28T01:06:23.727289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724224+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:06:23.724224+0000","last_peered":"2024-03-28T01:06:23.724224+0000","last_clean":"2024-03-28T01:06:23.724224+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:06:23.724224+0000","last_undegraded":"2024-03-28T01:06:23.724224+0000","last_fullsized":"2024-03-28T01:06:23.724224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722772+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:06:23.722772+0000","last_peered":"2024-03-28T01:06:23.722772+0000","last_clean":"2024-03-28T01:06:23.722772+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:06:23.722772+0000","last_undegraded":"2024-03-28T01:06:23.722772+0000","last_fullsized":"2024-03-28T01:06:23.722772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710748+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:06:24.710748+0000","last_peered":"2024-03-28T01:06:24.710748+0000","last_clean":"2024-03-28T01:06:24.710748+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:06:24.710748+0000","last_undegraded":"2024-03-28T01:06:24.710748+0000","last_fullsized":"2024-03-28T01:06:24.710748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854169+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:06:26.854169+0000","last_peered":"2024-03-28T01:06:26.854169+0000","last_clean":"2024-03-28T01:06:26.854169+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:06:26.854169+0000","last_undegraded":"2024-03-28T01:06:26.854169+0000","last_fullsized":"2024-03-28T01:06:26.854169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856921+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:06:26.856921+0000","last_peered":"2024-03-28T01:06:26.856921+0000","last_clean":"2024-03-28T01:06:26.856921+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:06:26.856921+0000","last_undegraded":"2024-03-28T01:06:26.856921+0000","last_fullsized":"2024-03-28T01:06:26.856921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261345+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:06:26.261345+0000","last_peered":"2024-03-28T01:06:26.261345+0000","last_clean":"2024-03-28T01:06:26.261345+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:06:26.261345+0000","last_undegraded":"2024-03-28T01:06:26.261345+0000","last_fullsized":"2024-03-28T01:06:26.261345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857651+0000","last_change":"2024-03-28T01:06:22.743819+0000","last_active":"2024-03-28T01:06:26.857651+0000","last_peered":"2024-03-28T01:06:26.857651+0000","last_clean":"2024-03-28T01:06:26.857651+0000","last_became_active":"2024-03-28T01:06:22.742898+0000","last_became_peered":"2024-03-28T01:06:22.742898+0000","last_unstale":"2024-03-28T01:06:26.857651+0000","last_undegraded":"2024-03-28T01:06:26.857651+0000","last_fullsized":"2024-03-28T01:06:26.857651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717295+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:06:24.717295+0000","last_peered":"2024-03-28T01:06:24.717295+0000","last_clean":"2024-03-28T01:06:24.717295+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:06:24.717295+0000","last_undegraded":"2024-03-28T01:06:24.717295+0000","last_fullsized":"2024-03-28T01:06:24.717295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722929+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:06:23.722929+0000","last_peered":"2024-03-28T01:06:23.722929+0000","last_clean":"2024-03-28T01:06:23.722929+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:06:23.722929+0000","last_undegraded":"2024-03-28T01:06:23.722929+0000","last_fullsized":"2024-03-28T01:06:23.722929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854322+0000","last_change":"2024-03-28T01:06:22.749132+0000","last_active":"2024-03-28T01:06:26.854322+0000","last_peered":"2024-03-28T01:06:26.854322+0000","last_clean":"2024-03-28T01:06:26.854322+0000","last_became_active":"2024-03-28T01:06:22.748012+0000","last_became_peered":"2024-03-28T01:06:22.748012+0000","last_unstale":"2024-03-28T01:06:26.854322+0000","last_undegraded":"2024-03-28T01:06:26.854322+0000","last_fullsized":"2024-03-28T01:06:26.854322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724138+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:06:23.724138+0000","last_peered":"2024-03-28T01:06:23.724138+0000","last_clean":"2024-03-28T01:06:23.724138+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:06:23.724138+0000","last_undegraded":"2024-03-28T01:06:23.724138+0000","last_fullsized":"2024-03-28T01:06:23.724138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854193+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:06:26.854193+0000","last_peered":"2024-03-28T01:06:26.854193+0000","last_clean":"2024-03-28T01:06:26.854193+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:06:26.854193+0000","last_undegraded":"2024-03-28T01:06:26.854193+0000","last_fullsized":"2024-03-28T01:06:26.854193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724983+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:06:23.724983+0000","last_peered":"2024-03-28T01:06:23.724983+0000","last_clean":"2024-03-28T01:06:23.724983+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:06:23.724983+0000","last_undegraded":"2024-03-28T01:06:23.724983+0000","last_fullsized":"2024-03-28T01:06:23.724983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.718057+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:06:24.718057+0000","last_peered":"2024-03-28T01:06:24.718057+0000","last_clean":"2024-03-28T01:06:24.718057+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:06:24.718057+0000","last_undegraded":"2024-03-28T01:06:24.718057+0000","last_fullsized":"2024-03-28T01:06:24.718057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711534+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:06:24.711534+0000","last_peered":"2024-03-28T01:06:24.711534+0000","last_clean":"2024-03-28T01:06:24.711534+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:06:24.711534+0000","last_undegraded":"2024-03-28T01:06:24.711534+0000","last_fullsized":"2024-03-28T01:06:24.711534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725154+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:06:23.725154+0000","last_peered":"2024-03-28T01:06:23.725154+0000","last_clean":"2024-03-28T01:06:23.725154+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:06:23.725154+0000","last_undegraded":"2024-03-28T01:06:23.725154+0000","last_fullsized":"2024-03-28T01:06:23.725154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723116+0000","last_change":"2024-03-28T01:06:22.731272+0000","last_active":"2024-03-28T01:06:23.723116+0000","last_peered":"2024-03-28T01:06:23.723116+0000","last_clean":"2024-03-28T01:06:23.723116+0000","last_became_active":"2024-03-28T01:06:22.730261+0000","last_became_peered":"2024-03-28T01:06:22.730261+0000","last_unstale":"2024-03-28T01:06:23.723116+0000","last_undegraded":"2024-03-28T01:06:23.723116+0000","last_fullsized":"2024-03-28T01:06:23.723116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855545+0000","last_change":"2024-03-28T01:06:22.746590+0000","last_active":"2024-03-28T01:06:26.855545+0000","last_peered":"2024-03-28T01:06:26.855545+0000","last_clean":"2024-03-28T01:06:26.855545+0000","last_became_active":"2024-03-28T01:06:22.746299+0000","last_became_peered":"2024-03-28T01:06:22.746299+0000","last_unstale":"2024-03-28T01:06:26.855545+0000","last_undegraded":"2024-03-28T01:06:26.855545+0000","last_fullsized":"2024-03-28T01:06:26.855545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261649+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:06:26.261649+0000","last_peered":"2024-03-28T01:06:26.261649+0000","last_clean":"2024-03-28T01:06:26.261649+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:06:26.261649+0000","last_undegraded":"2024-03-28T01:06:26.261649+0000","last_fullsized":"2024-03-28T01:06:26.261649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717294+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:06:24.717294+0000","last_peered":"2024-03-28T01:06:24.717294+0000","last_clean":"2024-03-28T01:06:24.717294+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:06:24.717294+0000","last_undegraded":"2024-03-28T01:06:24.717294+0000","last_fullsized":"2024-03-28T01:06:24.717294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726504+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:06:23.726504+0000","last_peered":"2024-03-28T01:06:23.726504+0000","last_clean":"2024-03-28T01:06:23.726504+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:06:23.726504+0000","last_undegraded":"2024-03-28T01:06:23.726504+0000","last_fullsized":"2024-03-28T01:06:23.726504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724518+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:06:23.724518+0000","last_peered":"2024-03-28T01:06:23.724518+0000","last_clean":"2024-03-28T01:06:23.724518+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:06:23.724518+0000","last_undegraded":"2024-03-28T01:06:23.724518+0000","last_fullsized":"2024-03-28T01:06:23.724518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723265+0000","last_change":"2024-03-28T01:06:22.717986+0000","last_active":"2024-03-28T01:06:23.723265+0000","last_peered":"2024-03-28T01:06:23.723265+0000","last_clean":"2024-03-28T01:06:23.723265+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:06:23.723265+0000","last_undegraded":"2024-03-28T01:06:23.723265+0000","last_fullsized":"2024-03-28T01:06:23.723265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717592+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:06:24.717592+0000","last_peered":"2024-03-28T01:06:24.717592+0000","last_clean":"2024-03-28T01:06:24.717592+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:06:24.717592+0000","last_undegraded":"2024-03-28T01:06:24.717592+0000","last_fullsized":"2024-03-28T01:06:24.717592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724278+0000","last_change":"2024-03-28T01:06:21.721278+0000","last_active":"2024-03-28T01:06:23.724278+0000","last_peered":"2024-03-28T01:06:23.724278+0000","last_clean":"2024-03-28T01:06:23.724278+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:06:23.724278+0000","last_undegraded":"2024-03-28T01:06:23.724278+0000","last_fullsized":"2024-03-28T01:06:23.724278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723815+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:06:23.723815+0000","last_peered":"2024-03-28T01:06:23.723815+0000","last_clean":"2024-03-28T01:06:23.723815+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:06:23.723815+0000","last_undegraded":"2024-03-28T01:06:23.723815+0000","last_fullsized":"2024-03-28T01:06:23.723815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854498+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:06:26.854498+0000","last_peered":"2024-03-28T01:06:26.854498+0000","last_clean":"2024-03-28T01:06:26.854498+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:06:26.854498+0000","last_undegraded":"2024-03-28T01:06:26.854498+0000","last_fullsized":"2024-03-28T01:06:26.854498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710729+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:06:24.710729+0000","last_peered":"2024-03-28T01:06:24.710729+0000","last_clean":"2024-03-28T01:06:24.710729+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:06:24.710729+0000","last_undegraded":"2024-03-28T01:06:24.710729+0000","last_fullsized":"2024-03-28T01:06:24.710729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717704+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:06:24.717704+0000","last_peered":"2024-03-28T01:06:24.717704+0000","last_clean":"2024-03-28T01:06:24.717704+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:06:24.717704+0000","last_undegraded":"2024-03-28T01:06:24.717704+0000","last_fullsized":"2024-03-28T01:06:24.717704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855761+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:06:26.855761+0000","last_peered":"2024-03-28T01:06:26.855761+0000","last_clean":"2024-03-28T01:06:26.855761+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:06:26.855761+0000","last_undegraded":"2024-03-28T01:06:26.855761+0000","last_fullsized":"2024-03-28T01:06:26.855761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855830+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:06:26.855830+0000","last_peered":"2024-03-28T01:06:26.855830+0000","last_clean":"2024-03-28T01:06:26.855830+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:06:26.855830+0000","last_undegraded":"2024-03-28T01:06:26.855830+0000","last_fullsized":"2024-03-28T01:06:26.855830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724058+0000","last_change":"2024-03-28T01:06:22.743892+0000","last_active":"2024-03-28T01:06:23.724058+0000","last_peered":"2024-03-28T01:06:23.724058+0000","last_clean":"2024-03-28T01:06:23.724058+0000","last_became_active":"2024-03-28T01:06:22.743113+0000","last_became_peered":"2024-03-28T01:06:22.743113+0000","last_unstale":"2024-03-28T01:06:23.724058+0000","last_undegraded":"2024-03-28T01:06:23.724058+0000","last_fullsized":"2024-03-28T01:06:23.724058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711550+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:06:24.711550+0000","last_peered":"2024-03-28T01:06:24.711550+0000","last_clean":"2024-03-28T01:06:24.711550+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:06:24.711550+0000","last_undegraded":"2024-03-28T01:06:24.711550+0000","last_fullsized":"2024-03-28T01:06:24.711550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727568+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:06:23.727568+0000","last_peered":"2024-03-28T01:06:23.727568+0000","last_clean":"2024-03-28T01:06:23.727568+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:06:23.727568+0000","last_undegraded":"2024-03-28T01:06:23.727568+0000","last_fullsized":"2024-03-28T01:06:23.727568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717385+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:06:24.717385+0000","last_peered":"2024-03-28T01:06:24.717385+0000","last_clean":"2024-03-28T01:06:24.717385+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:06:24.717385+0000","last_undegraded":"2024-03-28T01:06:24.717385+0000","last_fullsized":"2024-03-28T01:06:24.717385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856677+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:06:26.856677+0000","last_peered":"2024-03-28T01:06:26.856677+0000","last_clean":"2024-03-28T01:06:26.856677+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:06:26.856677+0000","last_undegraded":"2024-03-28T01:06:26.856677+0000","last_fullsized":"2024-03-28T01:06:26.856677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723346+0000","last_change":"2024-03-28T01:06:21.727305+0000","last_active":"2024-03-28T01:06:23.723346+0000","last_peered":"2024-03-28T01:06:23.723346+0000","last_clean":"2024-03-28T01:06:23.723346+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:06:23.723346+0000","last_undegraded":"2024-03-28T01:06:23.723346+0000","last_fullsized":"2024-03-28T01:06:23.723346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722951+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:06:23.722951+0000","last_peered":"2024-03-28T01:06:23.722951+0000","last_clean":"2024-03-28T01:06:23.722951+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:06:23.722951+0000","last_undegraded":"2024-03-28T01:06:23.722951+0000","last_fullsized":"2024-03-28T01:06:23.722951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854839+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:06:26.854839+0000","last_peered":"2024-03-28T01:06:26.854839+0000","last_clean":"2024-03-28T01:06:26.854839+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:06:26.854839+0000","last_undegraded":"2024-03-28T01:06:26.854839+0000","last_fullsized":"2024-03-28T01:06:26.854839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857353+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:06:26.857353+0000","last_peered":"2024-03-28T01:06:26.857353+0000","last_clean":"2024-03-28T01:06:26.857353+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:06:26.857353+0000","last_undegraded":"2024-03-28T01:06:26.857353+0000","last_fullsized":"2024-03-28T01:06:26.857353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717022+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:06:24.717022+0000","last_peered":"2024-03-28T01:06:24.717022+0000","last_clean":"2024-03-28T01:06:24.717022+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:06:24.717022+0000","last_undegraded":"2024-03-28T01:06:24.717022+0000","last_fullsized":"2024-03-28T01:06:24.717022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717552+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:06:24.717552+0000","last_peered":"2024-03-28T01:06:24.717552+0000","last_clean":"2024-03-28T01:06:24.717552+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:06:24.717552+0000","last_undegraded":"2024-03-28T01:06:24.717552+0000","last_fullsized":"2024-03-28T01:06:24.717552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261265+0000","last_change":"2024-03-28T01:06:21.719466+0000","last_active":"2024-03-28T01:06:26.261265+0000","last_peered":"2024-03-28T01:06:26.261265+0000","last_clean":"2024-03-28T01:06:26.261265+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:06:26.261265+0000","last_undegraded":"2024-03-28T01:06:26.261265+0000","last_fullsized":"2024-03-28T01:06:26.261265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856277+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:06:26.856277+0000","last_peered":"2024-03-28T01:06:26.856277+0000","last_clean":"2024-03-28T01:06:26.856277+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:06:26.856277+0000","last_undegraded":"2024-03-28T01:06:26.856277+0000","last_fullsized":"2024-03-28T01:06:26.856277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727523+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:06:23.727523+0000","last_peered":"2024-03-28T01:06:23.727523+0000","last_clean":"2024-03-28T01:06:23.727523+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:06:23.727523+0000","last_undegraded":"2024-03-28T01:06:23.727523+0000","last_fullsized":"2024-03-28T01:06:23.727523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856873+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:06:26.856873+0000","last_peered":"2024-03-28T01:06:26.856873+0000","last_clean":"2024-03-28T01:06:26.856873+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:06:26.856873+0000","last_undegraded":"2024-03-28T01:06:26.856873+0000","last_fullsized":"2024-03-28T01:06:26.856873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261460+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:06:26.261460+0000","last_peered":"2024-03-28T01:06:26.261460+0000","last_clean":"2024-03-28T01:06:26.261460+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:06:26.261460+0000","last_undegraded":"2024-03-28T01:06:26.261460+0000","last_fullsized":"2024-03-28T01:06:26.261460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.858459+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:06:26.858459+0000","last_peered":"2024-03-28T01:06:26.858459+0000","last_clean":"2024-03-28T01:06:26.858459+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:06:26.858459+0000","last_undegraded":"2024-03-28T01:06:26.858459+0000","last_fullsized":"2024-03-28T01:06:26.858459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711878+0000","last_change":"2024-03-28T01:06:21.718159+0000","last_active":"2024-03-28T01:06:24.711878+0000","last_peered":"2024-03-28T01:06:24.711878+0000","last_clean":"2024-03-28T01:06:24.711878+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:06:24.711878+0000","last_undegraded":"2024-03-28T01:06:24.711878+0000","last_fullsized":"2024-03-28T01:06:24.711878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855554+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:06:26.855554+0000","last_peered":"2024-03-28T01:06:26.855554+0000","last_clean":"2024-03-28T01:06:26.855554+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:06:26.855554+0000","last_undegraded":"2024-03-28T01:06:26.855554+0000","last_fullsized":"2024-03-28T01:06:26.855554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261512+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:06:26.261512+0000","last_peered":"2024-03-28T01:06:26.261512+0000","last_clean":"2024-03-28T01:06:26.261512+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:06:26.261512+0000","last_undegraded":"2024-03-28T01:06:26.261512+0000","last_fullsized":"2024-03-28T01:06:26.261512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856940+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:06:26.856940+0000","last_peered":"2024-03-28T01:06:26.856940+0000","last_clean":"2024-03-28T01:06:26.856940+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:06:26.856940+0000","last_undegraded":"2024-03-28T01:06:26.856940+0000","last_fullsized":"2024-03-28T01:06:26.856940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.856017+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:06:26.856017+0000","last_peered":"2024-03-28T01:06:26.856017+0000","last_clean":"2024-03-28T01:06:26.856017+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:06:26.856017+0000","last_undegraded":"2024-03-28T01:06:26.856017+0000","last_fullsized":"2024-03-28T01:06:26.856017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.854326+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:06:26.854326+0000","last_peered":"2024-03-28T01:06:26.854326+0000","last_clean":"2024-03-28T01:06:26.854326+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:06:26.854326+0000","last_undegraded":"2024-03-28T01:06:26.854326+0000","last_fullsized":"2024-03-28T01:06:26.854326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724610+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:06:23.724610+0000","last_peered":"2024-03-28T01:06:23.724610+0000","last_clean":"2024-03-28T01:06:23.724610+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:06:23.724610+0000","last_undegraded":"2024-03-28T01:06:23.724610+0000","last_fullsized":"2024-03-28T01:06:23.724610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857175+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:06:26.857175+0000","last_peered":"2024-03-28T01:06:26.857175+0000","last_clean":"2024-03-28T01:06:26.857175+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:06:26.857175+0000","last_undegraded":"2024-03-28T01:06:26.857175+0000","last_fullsized":"2024-03-28T01:06:26.857175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.722723+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:06:23.722723+0000","last_peered":"2024-03-28T01:06:23.722723+0000","last_clean":"2024-03-28T01:06:23.722723+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:06:23.722723+0000","last_undegraded":"2024-03-28T01:06:23.722723+0000","last_fullsized":"2024-03-28T01:06:23.722723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.261507+0000","last_change":"2024-03-28T01:06:21.720753+0000","last_active":"2024-03-28T01:06:26.261507+0000","last_peered":"2024-03-28T01:06:26.261507+0000","last_clean":"2024-03-28T01:06:26.261507+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:06:26.261507+0000","last_undegraded":"2024-03-28T01:06:26.261507+0000","last_fullsized":"2024-03-28T01:06:26.261507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710942+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:06:24.710942+0000","last_peered":"2024-03-28T01:06:24.710942+0000","last_clean":"2024-03-28T01:06:24.710942+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:06:24.710942+0000","last_undegraded":"2024-03-28T01:06:24.710942+0000","last_fullsized":"2024-03-28T01:06:24.710942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855089+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:06:26.855089+0000","last_peered":"2024-03-28T01:06:26.855089+0000","last_clean":"2024-03-28T01:06:26.855089+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:06:26.855089+0000","last_undegraded":"2024-03-28T01:06:26.855089+0000","last_fullsized":"2024-03-28T01:06:26.855089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.727032+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:06:23.727032+0000","last_peered":"2024-03-28T01:06:23.727032+0000","last_clean":"2024-03-28T01:06:23.727032+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:06:23.727032+0000","last_undegraded":"2024-03-28T01:06:23.727032+0000","last_fullsized":"2024-03-28T01:06:23.727032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711510+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:06:24.711510+0000","last_peered":"2024-03-28T01:06:24.711510+0000","last_clean":"2024-03-28T01:06:24.711510+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:06:24.711510+0000","last_undegraded":"2024-03-28T01:06:24.711510+0000","last_fullsized":"2024-03-28T01:06:24.711510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855665+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:06:26.855665+0000","last_peered":"2024-03-28T01:06:26.855665+0000","last_clean":"2024-03-28T01:06:26.855665+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:06:26.855665+0000","last_undegraded":"2024-03-28T01:06:26.855665+0000","last_fullsized":"2024-03-28T01:06:26.855665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857875+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:06:26.857875+0000","last_peered":"2024-03-28T01:06:26.857875+0000","last_clean":"2024-03-28T01:06:26.857875+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:06:26.857875+0000","last_undegraded":"2024-03-28T01:06:26.857875+0000","last_fullsized":"2024-03-28T01:06:26.857875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711037+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:06:24.711037+0000","last_peered":"2024-03-28T01:06:24.711037+0000","last_clean":"2024-03-28T01:06:24.711037+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:06:24.711037+0000","last_undegraded":"2024-03-28T01:06:24.711037+0000","last_fullsized":"2024-03-28T01:06:24.711037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.715740+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:06:24.715740+0000","last_peered":"2024-03-28T01:06:24.715740+0000","last_clean":"2024-03-28T01:06:24.715740+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:06:24.715740+0000","last_undegraded":"2024-03-28T01:06:24.715740+0000","last_fullsized":"2024-03-28T01:06:24.715740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:34.313143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725201+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:06:23.725201+0000","last_peered":"2024-03-28T01:06:23.725201+0000","last_clean":"2024-03-28T01:06:23.725201+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:06:23.725201+0000","last_undegraded":"2024-03-28T01:06:23.725201+0000","last_fullsized":"2024-03-28T01:06:23.725201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717736+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:06:24.717736+0000","last_peered":"2024-03-28T01:06:24.717736+0000","last_clean":"2024-03-28T01:06:24.717736+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:06:24.717736+0000","last_undegraded":"2024-03-28T01:06:24.717736+0000","last_fullsized":"2024-03-28T01:06:24.717736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723864+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:06:23.723864+0000","last_peered":"2024-03-28T01:06:23.723864+0000","last_clean":"2024-03-28T01:06:23.723864+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:06:23.723864+0000","last_undegraded":"2024-03-28T01:06:23.723864+0000","last_fullsized":"2024-03-28T01:06:23.723864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855061+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:06:26.855061+0000","last_peered":"2024-03-28T01:06:26.855061+0000","last_clean":"2024-03-28T01:06:26.855061+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:06:26.855061+0000","last_undegraded":"2024-03-28T01:06:26.855061+0000","last_fullsized":"2024-03-28T01:06:26.855061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.726343+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:06:23.726343+0000","last_peered":"2024-03-28T01:06:23.726343+0000","last_clean":"2024-03-28T01:06:23.726343+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:06:23.726343+0000","last_undegraded":"2024-03-28T01:06:23.726343+0000","last_fullsized":"2024-03-28T01:06:23.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.710944+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:06:24.710944+0000","last_peered":"2024-03-28T01:06:24.710944+0000","last_clean":"2024-03-28T01:06:24.710944+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:06:24.710944+0000","last_undegraded":"2024-03-28T01:06:24.710944+0000","last_fullsized":"2024-03-28T01:06:24.710944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723935+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:06:23.723935+0000","last_peered":"2024-03-28T01:06:23.723935+0000","last_clean":"2024-03-28T01:06:23.723935+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:06:23.723935+0000","last_undegraded":"2024-03-28T01:06:23.723935+0000","last_fullsized":"2024-03-28T01:06:23.723935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.711041+0000","last_change":"2024-03-28T01:06:21.715183+0000","last_active":"2024-03-28T01:06:24.711041+0000","last_peered":"2024-03-28T01:06:24.711041+0000","last_clean":"2024-03-28T01:06:24.711041+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:06:24.711041+0000","last_undegraded":"2024-03-28T01:06:24.711041+0000","last_fullsized":"2024-03-28T01:06:24.711041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724403+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:06:23.724403+0000","last_peered":"2024-03-28T01:06:23.724403+0000","last_clean":"2024-03-28T01:06:23.724403+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:06:23.724403+0000","last_undegraded":"2024-03-28T01:06:23.724403+0000","last_fullsized":"2024-03-28T01:06:23.724403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857617+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:06:26.857617+0000","last_peered":"2024-03-28T01:06:26.857617+0000","last_clean":"2024-03-28T01:06:26.857617+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:06:26.857617+0000","last_undegraded":"2024-03-28T01:06:26.857617+0000","last_fullsized":"2024-03-28T01:06:26.857617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:24.717267+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:06:24.717267+0000","last_peered":"2024-03-28T01:06:24.717267+0000","last_clean":"2024-03-28T01:06:24.717267+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:06:24.717267+0000","last_undegraded":"2024-03-28T01:06:24.717267+0000","last_fullsized":"2024-03-28T01:06:24.717267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855244+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:06:26.855244+0000","last_peered":"2024-03-28T01:06:26.855244+0000","last_clean":"2024-03-28T01:06:26.855244+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:06:26.855244+0000","last_undegraded":"2024-03-28T01:06:26.855244+0000","last_fullsized":"2024-03-28T01:06:26.855244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857157+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:06:26.857157+0000","last_peered":"2024-03-28T01:06:26.857157+0000","last_clean":"2024-03-28T01:06:26.857157+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:06:26.857157+0000","last_undegraded":"2024-03-28T01:06:26.857157+0000","last_fullsized":"2024-03-28T01:06:26.857157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.855081+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:06:26.855081+0000","last_peered":"2024-03-28T01:06:26.855081+0000","last_clean":"2024-03-28T01:06:26.855081+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:06:26.855081+0000","last_undegraded":"2024-03-28T01:06:26.855081+0000","last_fullsized":"2024-03-28T01:06:26.855081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:26.857206+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:06:26.857206+0000","last_peered":"2024-03-28T01:06:26.857206+0000","last_clean":"2024-03-28T01:06:26.857206+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:06:26.857206+0000","last_undegraded":"2024-03-28T01:06:26.857206+0000","last_fullsized":"2024-03-28T01:06:26.857206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.723980+0000","last_change":"2024-03-28T01:06:22.742813+0000","last_active":"2024-03-28T01:06:23.723980+0000","last_peered":"2024-03-28T01:06:23.723980+0000","last_clean":"2024-03-28T01:06:23.723980+0000","last_became_active":"2024-03-28T01:06:22.736152+0000","last_became_peered":"2024-03-28T01:06:22.736152+0000","last_unstale":"2024-03-28T01:06:23.723980+0000","last_undegraded":"2024-03-28T01:06:23.723980+0000","last_fullsized":"2024-03-28T01:06:23.723980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725147+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:06:23.725147+0000","last_peered":"2024-03-28T01:06:23.725147+0000","last_clean":"2024-03-28T01:06:23.725147+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:06:23.725147+0000","last_undegraded":"2024-03-28T01:06:23.725147+0000","last_fullsized":"2024-03-28T01:06:23.725147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:32.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.725002+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:06:23.725002+0000","last_peered":"2024-03-28T01:06:23.725002+0000","last_clean":"2024-03-28T01:06:23.725002+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:06:23.725002+0000","last_undegraded":"2024-03-28T01:06:23.725002+0000","last_fullsized":"2024-03-28T01:06:23.725002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T01:06:23.724107+0000","last_change":"2024-03-28T01:06:21.727858+0000","last_active":"2024-03-28T01:06:23.724107+0000","last_peered":"2024-03-28T01:06:23.724107+0000","last_clean":"2024-03-28T01:06:23.724107+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:06:23.724107+0000","last_undegraded":"2024-03-28T01:06:23.724107+0000","last_fullsized":"2024-03-28T01:06:23.724107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:15:55.049218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:06:33.583 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-28T01:06:33.583 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-28T01:06:33.583 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-28T01:06:33.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T01:06:33.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.832+0000 7fddcee00640 1 Processor -- start 2024-03-28T01:06:33.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.833+0000 7fddcee00640 1 -- start start 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddcee00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fddc806d3e0 0x7fddc80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddcee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc80b2d00 0x7fddc80b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddcee00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc806d7c0 con 0x7fddc813eea0 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddcee00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc80b10f0 con 0x7fddc806d3e0 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddcee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80b0aa0 con 0x7fddc80b2d00 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddcce00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fddc813eea0 0x7fddc813f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55564/0 (socket says 172.21.15.33:55564) 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddcce00640 1 -- 172.21.15.33:0/3476461170 learned_addr learned my addr 172.21.15.33:0/3476461170 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:33.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddc7e00640 1 --2- 172.21.15.33:0/3476461170 >> v2:172.21.15.33:3300/0 conn(0x7fddc806d3e0 0x7fddc80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.834+0000 7fddc7400640 1 --2- 172.21.15.33:0/3476461170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc80b2d00 0x7fddc80b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.835+0000 7fddc6a00640 1 -- 172.21.15.33:0/3476461170 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725203200 0 0) 0x7fddc806d7c0 con 0x7fddc813eea0 2024-03-28T01:06:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.835+0000 7fddc6a00640 1 -- 172.21.15.33:0/3476461170 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddac003520 con 0x7fddc813eea0 2024-03-28T01:06:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.835+0000 7fddc7e00640 1 -- 172.21.15.33:0/3476461170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc80b2d00 msgr2=0x7fddc80b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.835+0000 7fddc7e00640 1 --2- 172.21.15.33:0/3476461170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc80b2d00 0x7fddc80b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.835+0000 7fddc7e00640 1 -- 172.21.15.33:0/3476461170 >> v1:172.21.15.33:6789/0 conn(0x7fddc813eea0 legacy=0x7fddc813f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.835+0000 7fddc7e00640 1 -- 172.21.15.33:0/3476461170 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80b0d80 con 0x7fddc806d3e0 2024-03-28T01:06:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.836+0000 7fddc7e00640 1 --2- 172.21.15.33:0/3476461170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc806d3e0 0x7fddc80b4a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fddb80097a0 tx=0x7fddb802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8afc87976697df7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.836+0000 7fddc6a00640 1 -- 172.21.15.33:0/3476461170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80315a0 con 0x7fddc806d3e0 2024-03-28T01:06:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.836+0000 7fddc6a00640 1 -- 172.21.15.33:0/3476461170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8037070 con 0x7fddc806d3e0 2024-03-28T01:06:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.836+0000 7fddc6a00640 1 -- 172.21.15.33:0/3476461170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8031940 con 0x7fddc806d3e0 2024-03-28T01:06:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 -- 172.21.15.33:0/3476461170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc806d3e0 msgr2=0x7fddc80b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 --2- 172.21.15.33:0/3476461170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc806d3e0 0x7fddc80b4a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fddb80097a0 tx=0x7fddb802b840 comp rx=0 tx=0).stop 2024-03-28T01:06:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 -- 172.21.15.33:0/3476461170 shutdown_connections 2024-03-28T01:06:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 --2- 172.21.15.33:0/3476461170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc80b2d00 0x7fddc80b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 --2- 172.21.15.33:0/3476461170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc806d3e0 0x7fddc80b4a60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 -- 172.21.15.33:0/3476461170 >> 172.21.15.33:0/3476461170 conn(0x7fddc806b030 msgr2=0x7fddc806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 -- 172.21.15.33:0/3476461170 shutdown_connections 2024-03-28T01:06:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.837+0000 7fddcee00640 1 -- 172.21.15.33:0/3476461170 wait complete. 2024-03-28T01:06:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.839+0000 7fddcee00640 1 Processor -- start 2024-03-28T01:06:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.839+0000 7fddcee00640 1 -- start start 2024-03-28T01:06:33.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc806d3e0 0x7fddc809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcee00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc80b2d00 0x7fddc809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddc813eea0 0x7fddc8071e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcee00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80b0150 con 0x7fddc80b2d00 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80affd0 con 0x7fddc813eea0 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc814fab0 con 0x7fddc806d3e0 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc80b2d00 0x7fddc809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc80b2d00 0x7fddc809aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40214/0 (socket says 172.21.15.33:40214) 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 -- 172.21.15.33:0/2401682869 learned_addr learned my addr 172.21.15.33:0/2401682869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7e00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc806d3e0 0x7fddc809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcce00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddc813eea0 0x7fddc8071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 -- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc806d3e0 msgr2=0x7fddc809cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc806d3e0 0x7fddc809cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 -- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddc813eea0 msgr2=0x7fddc8071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:33.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddc813eea0 0x7fddc8071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:33.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddc7400640 1 -- 172.21.15.33:0/2401682869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc8146b80 con 0x7fddc80b2d00 2024-03-28T01:06:33.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.840+0000 7fddcce00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddc813eea0 0x7fddc8071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:33.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.841+0000 7fddc7400640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc80b2d00 0x7fddc809aa50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e9b0 tx=0x7fddb400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:33.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.841+0000 7fddbfe00640 1 -- 172.21.15.33:0/2401682869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4017070 con 0x7fddc80b2d00 2024-03-28T01:06:33.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.841+0000 7fddbfe00640 1 -- 172.21.15.33:0/2401682869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb400c9a0 con 0x7fddc80b2d00 2024-03-28T01:06:33.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.841+0000 7fddbfe00640 1 -- 172.21.15.33:0/2401682869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb400cd90 con 0x7fddc80b2d00 2024-03-28T01:06:33.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.841+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc809dfd0 con 0x7fddc80b2d00 2024-03-28T01:06:33.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.841+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc809baf0 con 0x7fddc80b2d00 2024-03-28T01:06:33.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.842+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc809e4f0 con 0x7fddc80b2d00 2024-03-28T01:06:33.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.843+0000 7fddbfe00640 1 -- 172.21.15.33:0/2401682869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fddb401a070 con 0x7fddc80b2d00 2024-03-28T01:06:33.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.844+0000 7fddbfe00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdda806bbd0 0x7fdda806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:33.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.844+0000 7fddbfe00640 1 -- 172.21.15.33:0/2401682869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb408ad70 con 0x7fddc80b2d00 2024-03-28T01:06:33.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.844+0000 7fddc7e00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdda806bbd0 0x7fdda806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:33.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.845+0000 7fddc7e00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdda806bbd0 0x7fdda806e090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fddb8036000 tx=0x7fddb803d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:33.848+0000 7fddbfe00640 1 -- 172.21.15.33:0/2401682869 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fddb40598f0 con 0x7fddc80b2d00 2024-03-28T01:06:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.204+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fddc8073510 con 0x7fddc80b2d00 2024-03-28T01:06:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.205+0000 7fddbfe00640 1 -- 172.21.15.33:0/2401682869 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fddc8073510 con 0x7fddc80b2d00 2024-03-28T01:06:34.208 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:34.208 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdda806bbd0 msgr2=0x7fdda806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdda806bbd0 0x7fdda806e090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fddb8036000 tx=0x7fddb803d040 comp rx=0 tx=0).stop 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc80b2d00 msgr2=0x7fddc809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc80b2d00 0x7fddc809aa50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e9b0 tx=0x7fddb400ee70 comp rx=0 tx=0).stop 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 shutdown_connections 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fdda806bbd0 0x7fdda806e090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddc813eea0 0x7fddc8071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc80b2d00 0x7fddc809aa50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 --2- 172.21.15.33:0/2401682869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddc806d3e0 0x7fddc809cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 >> 172.21.15.33:0/2401682869 conn(0x7fddc806b030 msgr2=0x7fddc80b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 shutdown_connections 2024-03-28T01:06:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.208+0000 7fddcee00640 1 -- 172.21.15.33:0/2401682869 wait complete. 2024-03-28T01:06:34.230 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-28T01:06:34.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:34.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.450+0000 7fabad600640 1 Processor -- start 2024-03-28T01:06:34.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.451+0000 7fabad600640 1 -- start start 2024-03-28T01:06:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7fabad600640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faba80b2130 0x7faba813edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7fabad600640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7faba813f330 0x7faba81485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7fabad600640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba80b2510 con 0x7faba8148b20 2024-03-28T01:06:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7fabad600640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8148f30 con 0x7faba80b2130 2024-03-28T01:06:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7fabad600640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba80b4670 con 0x7faba813f330 2024-03-28T01:06:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7fabacc00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faba8148b20 0x7faba8144f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55574/0 (socket says 172.21.15.33:55574) 2024-03-28T01:06:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7fabacc00640 1 -- 172.21.15.33:0/2526811097 learned_addr learned my addr 172.21.15.33:0/2526811097 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7faba7e00640 1 --2- 172.21.15.33:0/2526811097 >> v2:172.21.15.33:3300/0 conn(0x7faba80b2130 0x7faba813edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.452+0000 7faba7400640 1 --2- 172.21.15.33:0/2526811097 >> v2:172.21.15.194:3300/0 conn(0x7faba813f330 0x7faba81485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.453+0000 7faba6a00640 1 -- 172.21.15.33:0/2526811097 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011921520 0 0) 0x7faba80b2510 con 0x7faba8148b20 2024-03-28T01:06:34.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.453+0000 7faba6a00640 1 -- 172.21.15.33:0/2526811097 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab8c003520 con 0x7faba8148b20 2024-03-28T01:06:34.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.453+0000 7faba7e00640 1 -- 172.21.15.33:0/2526811097 >> v2:172.21.15.194:3300/0 conn(0x7faba813f330 msgr2=0x7faba81485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:34.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.453+0000 7faba7e00640 1 --2- 172.21.15.33:0/2526811097 >> v2:172.21.15.194:3300/0 conn(0x7faba813f330 0x7faba81485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.453+0000 7faba7e00640 1 -- 172.21.15.33:0/2526811097 >> v1:172.21.15.33:6789/0 conn(0x7faba8148b20 legacy=0x7faba8144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.453+0000 7faba7e00640 1 -- 172.21.15.33:0/2526811097 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba806bef0 con 0x7faba80b2130 2024-03-28T01:06:34.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.453+0000 7faba7400640 1 --2- 172.21.15.33:0/2526811097 >> v2:172.21.15.194:3300/0 conn(0x7faba813f330 0x7faba81485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:34.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.454+0000 7faba7e00640 1 --2- 172.21.15.33:0/2526811097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba80b2130 0x7faba813edf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faba806da10 tx=0x7fab9802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a3f84ba7761be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.454+0000 7faba6a00640 1 -- 172.21.15.33:0/2526811097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98039070 con 0x7faba80b2130 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.454+0000 7faba6a00640 1 -- 172.21.15.33:0/2526811097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9802cb60 con 0x7faba80b2130 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.454+0000 7faba6a00640 1 -- 172.21.15.33:0/2526811097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98034440 con 0x7faba80b2130 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 -- 172.21.15.33:0/2526811097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba80b2130 msgr2=0x7faba813edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 --2- 172.21.15.33:0/2526811097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba80b2130 0x7faba813edf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faba806da10 tx=0x7fab9802bdb0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 -- 172.21.15.33:0/2526811097 shutdown_connections 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 --2- 172.21.15.33:0/2526811097 >> v2:172.21.15.194:3300/0 conn(0x7faba813f330 0x7faba81485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 --2- 172.21.15.33:0/2526811097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba80b2130 0x7faba813edf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 -- 172.21.15.33:0/2526811097 >> 172.21.15.33:0/2526811097 conn(0x7faba80b89c0 msgr2=0x7faba80b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 -- 172.21.15.33:0/2526811097 shutdown_connections 2024-03-28T01:06:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.455+0000 7fabad600640 1 -- 172.21.15.33:0/2526811097 wait complete. 2024-03-28T01:06:34.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.458+0000 7fabad600640 1 Processor -- start 2024-03-28T01:06:34.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.458+0000 7fabad600640 1 -- start start 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.459+0000 7fabad600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faba80b2130 0x7faba80aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.459+0000 7fabad600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faba813f330 0x7faba80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.459+0000 7fabad600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8148b20 0x7faba80b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.459+0000 7fabad600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80b4ec0 con 0x7faba8148b20 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.459+0000 7fabad600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba806b6d0 con 0x7faba813f330 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.459+0000 7fabad600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba8072d20 con 0x7faba80b2130 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8148b20 0x7faba80b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8148b20 0x7faba80b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40238/0 (socket says 172.21.15.33:40238) 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 -- 172.21.15.33:0/1210020528 learned_addr learned my addr 172.21.15.33:0/1210020528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7faba7e00640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faba80b2130 0x7faba80aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7faba7400640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faba813f330 0x7faba80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 -- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faba80b2130 msgr2=0x7faba80aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faba80b2130 0x7faba80aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 -- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faba813f330 msgr2=0x7faba80b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faba813f330 0x7faba80b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 -- 172.21.15.33:0/1210020528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80b19a0 con 0x7faba8148b20 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.460+0000 7fabacc00640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8148b20 0x7faba80b7ee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faba000c390 tx=0x7faba000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.461+0000 7fab9fe00640 1 -- 172.21.15.33:0/1210020528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0017070 con 0x7faba8148b20 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.461+0000 7fab9fe00640 1 -- 172.21.15.33:0/1210020528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba0005ce0 con 0x7faba8148b20 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.461+0000 7fab9fe00640 1 -- 172.21.15.33:0/1210020528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0007310 con 0x7faba8148b20 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.461+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba80b1c80 con 0x7faba8148b20 2024-03-28T01:06:34.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.461+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba81d1ca0 con 0x7faba8148b20 2024-03-28T01:06:34.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.462+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba8075310 con 0x7faba8148b20 2024-03-28T01:06:34.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.463+0000 7fab9fe00640 1 -- 172.21.15.33:0/1210020528 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faba001a070 con 0x7faba8148b20 2024-03-28T01:06:34.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.465+0000 7fab9fe00640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fab8406bbd0 0x7fab8406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.465+0000 7fab9fe00640 1 -- 172.21.15.33:0/1210020528 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faba0089eb0 con 0x7faba8148b20 2024-03-28T01:06:34.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.465+0000 7faba7e00640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fab8406bbd0 0x7fab8406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.466+0000 7faba7e00640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fab8406bbd0 0x7fab8406e090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab98036000 tx=0x7fab98030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:34.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.468+0000 7fab9fe00640 1 -- 172.21.15.33:0/1210020528 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7faba0058990 con 0x7faba8148b20 2024-03-28T01:06:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.671+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faba806b470 con 0x7faba8148b20 2024-03-28T01:06:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.673+0000 7fab9fe00640 1 -- 172.21.15.33:0/1210020528 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7faba0058290 con 0x7faba8148b20 2024-03-28T01:06:34.676 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:34.676 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:23.710537+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fab8406bbd0 msgr2=0x7fab8406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fab8406bbd0 0x7fab8406e090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab98036000 tx=0x7fab98030000 comp rx=0 tx=0).stop 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8148b20 msgr2=0x7faba80b7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8148b20 0x7faba80b7ee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faba000c390 tx=0x7faba000c850 comp rx=0 tx=0).stop 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 shutdown_connections 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fab8406bbd0 0x7fab8406e090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8148b20 0x7faba80b7ee0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faba813f330 0x7faba80b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 --2- 172.21.15.33:0/1210020528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faba80b2130 0x7faba80aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 >> 172.21.15.33:0/1210020528 conn(0x7faba80b89c0 msgr2=0x7faba813b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 shutdown_connections 2024-03-28T01:06:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.676+0000 7fabad600640 1 -- 172.21.15.33:0/1210020528 wait complete. 2024-03-28T01:06:34.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T01:06:10.462126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T01:06:20.177809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T01:06:21.198872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T01:06:34.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T01:06:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.901+0000 7fc9af200640 1 Processor -- start 2024-03-28T01:06:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.902+0000 7fc9af200640 1 -- start start 2024-03-28T01:06:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.903+0000 7fc9af200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b00b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.903+0000 7fc9af200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fc9b0140cb0 0x7fc9b01410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.903+0000 7fc9af200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b00b6300 con 0x7fc9b00b4a30 2024-03-28T01:06:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.903+0000 7fc9af200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b00b6570 con 0x7fc9b0140cb0 2024-03-28T01:06:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.903+0000 7fc9af200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b00b5200 con 0x7fc9b00b9530 2024-03-28T01:06:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.904+0000 7fc9ad400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc9b00b4a30 0x7fc9b0147020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55580/0 (socket says 172.21.15.33:55580) 2024-03-28T01:06:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.904+0000 7fc9ad400640 1 -- 172.21.15.33:0/3246842542 learned_addr learned my addr 172.21.15.33:0/3246842542 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:34.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.904+0000 7fc9ade00640 1 --2- 172.21.15.33:0/3246842542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b00b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.904+0000 7fc9ae800640 1 --2- 172.21.15.33:0/3246842542 >> v2:172.21.15.194:3300/0 conn(0x7fc9b0140cb0 0x7fc9b01410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.904+0000 7fc9aca00640 1 -- 172.21.15.33:0/3246842542 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540836077 0 0) 0x7fc9b00b6300 con 0x7fc9b00b4a30 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.904+0000 7fc9aca00640 1 -- 172.21.15.33:0/3246842542 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc994003520 con 0x7fc9b00b4a30 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.905+0000 7fc9ade00640 1 -- 172.21.15.33:0/3246842542 >> v2:172.21.15.194:3300/0 conn(0x7fc9b0140cb0 msgr2=0x7fc9b01410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.905+0000 7fc9ade00640 1 --2- 172.21.15.33:0/3246842542 >> v2:172.21.15.194:3300/0 conn(0x7fc9b0140cb0 0x7fc9b01410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.905+0000 7fc9ade00640 1 -- 172.21.15.33:0/3246842542 >> v1:172.21.15.33:6789/0 conn(0x7fc9b00b4a30 legacy=0x7fc9b0147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.905+0000 7fc9aca00640 1 -- 172.21.15.33:0/3246842542 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 466119292 0 0) 0x7fc994003520 con 0x7fc9b00b4a30 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.905+0000 7fc9ade00640 1 -- 172.21.15.33:0/3246842542 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b00b4510 con 0x7fc9b00b9530 2024-03-28T01:06:34.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.906+0000 7fc9ade00640 1 --2- 172.21.15.33:0/3246842542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b00b6a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009de0 tx=0x7fc9a002bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=623f5909d0e14644 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:34.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.907+0000 7fc9aca00640 1 -- 172.21.15.33:0/3246842542 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0039070 con 0x7fc9b00b9530 2024-03-28T01:06:34.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.907+0000 7fc9aca00640 1 -- 172.21.15.33:0/3246842542 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a002c850 con 0x7fc9b00b9530 2024-03-28T01:06:34.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.907+0000 7fc9aca00640 1 -- 172.21.15.33:0/3246842542 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c001670 con 0x7fc9b00b9530 2024-03-28T01:06:34.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.907+0000 7fc9af200640 1 -- 172.21.15.33:0/3246842542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 msgr2=0x7fc9b00b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.907+0000 7fc9af200640 1 --2- 172.21.15.33:0/3246842542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b00b6a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009de0 tx=0x7fc9a002bd20 comp rx=0 tx=0).stop 2024-03-28T01:06:34.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.908+0000 7fc9af200640 1 -- 172.21.15.33:0/3246842542 shutdown_connections 2024-03-28T01:06:34.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.908+0000 7fc9af200640 1 --2- 172.21.15.33:0/3246842542 >> v2:172.21.15.194:3300/0 conn(0x7fc9b0140cb0 0x7fc9b01410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.908+0000 7fc9af200640 1 --2- 172.21.15.33:0/3246842542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b00b6a00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.908+0000 7fc9af200640 1 -- 172.21.15.33:0/3246842542 >> 172.21.15.33:0/3246842542 conn(0x7fc9b0072760 msgr2=0x7fc9b0072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:34.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.908+0000 7fc9af200640 1 -- 172.21.15.33:0/3246842542 shutdown_connections 2024-03-28T01:06:34.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.908+0000 7fc9af200640 1 -- 172.21.15.33:0/3246842542 wait complete. 2024-03-28T01:06:34.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.910+0000 7fc9af200640 1 Processor -- start 2024-03-28T01:06:34.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.910+0000 7fc9af200640 1 -- start start 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9af200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9b00b4a30 0x7fc9b0144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9af200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b01ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9af200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9b0140cb0 0x7fc9b01cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9af200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0141620 con 0x7fc9b00b4a30 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9af200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b00b4e90 con 0x7fc9b0140cb0 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9af200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01cfa50 con 0x7fc9b00b9530 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9b00b4a30 0x7fc9b0144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9b00b4a30 0x7fc9b0144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40250/0 (socket says 172.21.15.33:40250) 2024-03-28T01:06:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 -- 172.21.15.33:0/1043625928 learned_addr learned my addr 172.21.15.33:0/1043625928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 -- 172.21.15.33:0/1043625928 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 msgr2=0x7fc9b01ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b01ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 -- 172.21.15.33:0/1043625928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9b0140cb0 msgr2=0x7fc9b01cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9b0140cb0 0x7fc9b01cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.912+0000 7fc9ade00640 1 -- 172.21.15.33:0/1043625928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b01cf2e0 con 0x7fc9b00b4a30 2024-03-28T01:06:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.913+0000 7fc9ade00640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9b00b4a30 0x7fc9b0144700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0037040 tx=0x7fc9a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.913+0000 7fc9a6000640 1 -- 172.21.15.33:0/1043625928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0039050 con 0x7fc9b00b4a30 2024-03-28T01:06:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.913+0000 7fc9a6000640 1 -- 172.21.15.33:0/1043625928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0045070 con 0x7fc9b00b4a30 2024-03-28T01:06:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.913+0000 7fc9a6000640 1 -- 172.21.15.33:0/1043625928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a002ce60 con 0x7fc9b00b4a30 2024-03-28T01:06:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.913+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b01cfc80 con 0x7fc9b00b4a30 2024-03-28T01:06:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.913+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b01ceff0 con 0x7fc9b00b4a30 2024-03-28T01:06:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.915+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b0075310 con 0x7fc9b00b4a30 2024-03-28T01:06:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.916+0000 7fc9a6000640 1 -- 172.21.15.33:0/1043625928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc9a0040690 con 0x7fc9b00b4a30 2024-03-28T01:06:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.917+0000 7fc9a6000640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc99006bc00 0x7fc99006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.917+0000 7fc9a6000640 1 -- 172.21.15.33:0/1043625928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9a00ad300 con 0x7fc9b00b4a30 2024-03-28T01:06:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.918+0000 7fc9ad400640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc99006bc00 0x7fc99006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:34.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.918+0000 7fc9ad400640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc99006bc00 0x7fc99006e0c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc99c005880 tx=0x7fc99c001080 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:34.923+0000 7fc9a6000640 1 -- 172.21.15.33:0/1043625928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fc9a007bde0 con 0x7fc9b00b4a30 2024-03-28T01:06:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.125+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9b00b5fb0 con 0x7fc9b00b4a30 2024-03-28T01:06:35.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.126+0000 7fc9a6000640 1 -- 172.21.15.33:0/1043625928 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9a007b6e0 con 0x7fc9b00b4a30 2024-03-28T01:06:35.128 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc99006bc00 msgr2=0x7fc99006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc99006bc00 0x7fc99006e0c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc99c005880 tx=0x7fc99c001080 comp rx=0 tx=0).stop 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9b00b4a30 msgr2=0x7fc9b0144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9b00b4a30 0x7fc9b0144700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0037040 tx=0x7fc9a0004a40 comp rx=0 tx=0).stop 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 shutdown_connections 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc99006bc00 0x7fc99006e0c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9b0140cb0 0x7fc9b01cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9b00b9530 0x7fc9b01ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 --2- 172.21.15.33:0/1043625928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9b00b4a30 0x7fc9b0144700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 >> 172.21.15.33:0/1043625928 conn(0x7fc9b0072760 msgr2=0x7fc9b00bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 shutdown_connections 2024-03-28T01:06:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.129+0000 7fc9af200640 1 -- 172.21.15.33:0/1043625928 wait complete. 2024-03-28T01:06:35.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T01:06:35.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.369+0000 7fa88f400640 1 Processor -- start 2024-03-28T01:06:35.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.369+0000 7fa88f400640 1 -- start start 2024-03-28T01:06:35.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.372+0000 7fa88f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 0x7fa8880b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.372+0000 7fa88f400640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa888067420 con 0x7fa88809b160 2024-03-28T01:06:35.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.372+0000 7fa88f400640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8880ae8c0 con 0x7fa88809bb10 2024-03-28T01:06:35.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.372+0000 7fa88f400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880b5490 con 0x7fa888067040 2024-03-28T01:06:35.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa88d400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa88809b160 0x7fa888151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55588/0 (socket says 172.21.15.33:55588) 2024-03-28T01:06:35.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa88d400640 1 -- 172.21.15.33:0/2677599167 learned_addr learned my addr 172.21.15.33:0/2677599167 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:35.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa88ca00640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 0x7fa8880b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281298239 0 0) 0x7fa888067420 con 0x7fa88809b160 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa870003520 con 0x7fa88809b160 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473893045 0 0) 0x7fa870003520 con 0x7fa88809b160 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 msgr2=0x7fa8880b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.373+0000 7fa887400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 0x7fa8880b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 >> v1:172.21.15.194:6789/0 conn(0x7fa88809bb10 legacy=0x7fa88809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880b4e40 con 0x7fa88809b160 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7fa8800019f0 con 0x7fa88809b160 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 >> v1:172.21.15.33:6789/0 conn(0x7fa88809b160 legacy=0x7fa888151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 0x7fa8700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa870003f20 0x7fa870004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa870004d40 0x7fa8700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880b5490 con 0x7fa870004d40 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87000ba20 con 0x7fa870003f20 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87000bc20 con 0x7fa888067040 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa88d400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa870004d40 0x7fa8700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa88ca00640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 0x7fa8700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.374+0000 7fa887e00640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa870003f20 0x7fa870004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.375+0000 7fa88d400640 1 -- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 msgr2=0x7fa8700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.375+0000 7fa88d400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 0x7fa8700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.375+0000 7fa88d400640 1 -- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa870003f20 msgr2=0x7fa870004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.375+0000 7fa88d400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa870003f20 0x7fa870004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.375+0000 7fa88d400640 1 -- 172.21.15.33:0/2677599167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870003520 con 0x7fa870004d40 2024-03-28T01:06:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.375+0000 7fa88ca00640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa888067040 0x7fa8700039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.376+0000 7fa88d400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa870004d40 0x7fa8700051c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa88000a860 tx=0x7fa870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6870017d84e88a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.376+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000a480 con 0x7fa870004d40 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.376+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880034630 con 0x7fa870004d40 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.376+0000 7fa887400640 1 -- 172.21.15.33:0/2677599167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880034aa0 con 0x7fa870004d40 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.377+0000 7fa88f400640 1 -- 172.21.15.33:0/2677599167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa870004d40 msgr2=0x7fa8700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.377+0000 7fa88f400640 1 --2- 172.21.15.33:0/2677599167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa870004d40 0x7fa8700051c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa88000a860 tx=0x7fa870005830 comp rx=0 tx=0).stop 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.377+0000 7fa88d400640 1 -- 172.21.15.33:0/2677599167 reap_dead start 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.377+0000 7fa88f400640 1 -- 172.21.15.33:0/2677599167 shutdown_connections 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.377+0000 7fa88f400640 1 -- 172.21.15.33:0/2677599167 >> 172.21.15.33:0/2677599167 conn(0x7fa8880b5c40 msgr2=0x7fa8880b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.377+0000 7fa88f400640 1 -- 172.21.15.33:0/2677599167 shutdown_connections 2024-03-28T01:06:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.377+0000 7fa88f400640 1 -- 172.21.15.33:0/2677599167 wait complete. 2024-03-28T01:06:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.380+0000 7fa88f400640 1 Processor -- start 2024-03-28T01:06:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.380+0000 7fa88f400640 1 -- start start 2024-03-28T01:06:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.381+0000 7fa88f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa870004d40 0x7fa8881e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.381+0000 7fa88f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8881ceaa0 0x7fa8881cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.381+0000 7fa88f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8880b2320 0x7fa8880b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.381+0000 7fa88f400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880af1b0 con 0x7fa8880b2320 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.381+0000 7fa88f400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880af030 con 0x7fa8881ceaa0 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.381+0000 7fa88f400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880ae3e0 con 0x7fa870004d40 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.381+0000 7fa88d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8880b2320 0x7fa8880b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8880b2320 0x7fa8880b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40270/0 (socket says 172.21.15.33:40270) 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 -- 172.21.15.33:0/4248858762 learned_addr learned my addr 172.21.15.33:0/4248858762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88ca00640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa870004d40 0x7fa8881e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa887e00640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8881ceaa0 0x7fa8881cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 -- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa870004d40 msgr2=0x7fa8881e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa870004d40 0x7fa8881e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 -- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8881ceaa0 msgr2=0x7fa8881cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8881ceaa0 0x7fa8881cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 -- 172.21.15.33:0/4248858762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8881cf460 con 0x7fa8880b2320 2024-03-28T01:06:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.382+0000 7fa88d400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8880b2320 0x7fa8880b27a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa88002cc50 tx=0x7fa8881474c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.383+0000 7fa884c00640 1 -- 172.21.15.33:0/4248858762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880039040 con 0x7fa8880b2320 2024-03-28T01:06:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.383+0000 7fa884c00640 1 -- 172.21.15.33:0/4248858762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880043070 con 0x7fa8880b2320 2024-03-28T01:06:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.383+0000 7fa884c00640 1 -- 172.21.15.33:0/4248858762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000a480 con 0x7fa8880b2320 2024-03-28T01:06:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.383+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88809c6f0 con 0x7fa8880b2320 2024-03-28T01:06:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.383+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8881e1220 con 0x7fa8880b2320 2024-03-28T01:06:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.384+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa888075310 con 0x7fa8880b2320 2024-03-28T01:06:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.385+0000 7fa884c00640 1 -- 172.21.15.33:0/4248858762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa8800349e0 con 0x7fa8880b2320 2024-03-28T01:06:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.386+0000 7fa884c00640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa86806bc00 0x7fa86806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.386+0000 7fa884c00640 1 -- 172.21.15.33:0/4248858762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8800ac8a0 con 0x7fa8880b2320 2024-03-28T01:06:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.386+0000 7fa88ca00640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa86806bc00 0x7fa86806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.387+0000 7fa88ca00640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa86806bc00 0x7fa86806e0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009a90 tx=0x7fa87c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.390+0000 7fa884c00640 1 -- 172.21.15.33:0/4248858762 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa8881cf460 con 0x7fa8880b2320 2024-03-28T01:06:35.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.590+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa88809f2a0 con 0x7fa8880b2320 2024-03-28T01:06:35.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.591+0000 7fa884c00640 1 -- 172.21.15.33:0/4248858762 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa88007b380 con 0x7fa8880b2320 2024-03-28T01:06:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-03-28T01:06:35.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa86806bc00 msgr2=0x7fa86806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa86806bc00 0x7fa86806e0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009a90 tx=0x7fa87c008040 comp rx=0 tx=0).stop 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8880b2320 msgr2=0x7fa8880b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8880b2320 0x7fa8880b27a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa88002cc50 tx=0x7fa8881474c0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 shutdown_connections 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa86806bc00 0x7fa86806e0c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8880b2320 0x7fa8880b27a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8881ceaa0 0x7fa8881cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 --2- 172.21.15.33:0/4248858762 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa870004d40 0x7fa8881e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 >> 172.21.15.33:0/4248858762 conn(0x7fa8880b5c40 msgr2=0x7fa888146830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:35.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 shutdown_connections 2024-03-28T01:06:35.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.594+0000 7fa88f400640 1 -- 172.21.15.33:0/4248858762 wait complete. 2024-03-28T01:06:35.615 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T01:06:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.831+0000 7ff411c00640 1 Processor -- start 2024-03-28T01:06:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.832+0000 7ff411c00640 1 -- start start 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.833+0000 7ff411c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b9530 0x7ff40c0b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.833+0000 7ff411c00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff40c0b4a30 0x7ff40c147020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.833+0000 7ff411c00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7ff40c140cb0 0x7ff40c1410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.833+0000 7ff411c00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40c0b6570 con 0x7ff40c0b4a30 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.833+0000 7ff411c00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40c0b6300 con 0x7ff40c140cb0 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.833+0000 7ff411c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0b5200 con 0x7ff40c0b9530 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff40c0b4a30 0x7ff40c147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff40c0b4a30 0x7ff40c147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40286/0 (socket says 172.21.15.33:40286) 2024-03-28T01:06:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 -- 172.21.15.33:0/1008254707 learned_addr learned my addr 172.21.15.33:0/1008254707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff40be00640 1 --2- 172.21.15.33:0/1008254707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b9530 0x7ff40c0b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff411200640 1 --2- 172.21.15.33:0/1008254707 >> v2:172.21.15.194:3300/0 conn(0x7ff40c140cb0 0x7ff40c1410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 -- 172.21.15.33:0/1008254707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b9530 msgr2=0x7ff40c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 --2- 172.21.15.33:0/1008254707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b9530 0x7ff40c0b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 -- 172.21.15.33:0/1008254707 >> v2:172.21.15.194:3300/0 conn(0x7ff40c140cb0 msgr2=0x7ff40c1410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 --2- 172.21.15.33:0/1008254707 >> v2:172.21.15.194:3300/0 conn(0x7ff40c140cb0 0x7ff40c1410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.834+0000 7ff403400640 1 -- 172.21.15.33:0/1008254707 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0b4510 con 0x7ff40c0b4a30 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.835+0000 7ff403400640 1 --2- 172.21.15.33:0/1008254707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b4a30 0x7ff40c147020 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0b6f40 tx=0x7ff3f402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69bcd4b231a14e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.835+0000 7ff40b400640 1 -- 172.21.15.33:0/1008254707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40317c0 con 0x7ff40c0b4a30 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff40b400640 1 -- 172.21.15.33:0/1008254707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4037070 con 0x7ff40c0b4a30 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff40b400640 1 -- 172.21.15.33:0/1008254707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4031b80 con 0x7ff40c0b4a30 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 -- 172.21.15.33:0/1008254707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b4a30 msgr2=0x7ff40c147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 --2- 172.21.15.33:0/1008254707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b4a30 0x7ff40c147020 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0b6f40 tx=0x7ff3f402b860 comp rx=0 tx=0).stop 2024-03-28T01:06:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 -- 172.21.15.33:0/1008254707 shutdown_connections 2024-03-28T01:06:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 --2- 172.21.15.33:0/1008254707 >> v2:172.21.15.194:3300/0 conn(0x7ff40c140cb0 0x7ff40c1410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 --2- 172.21.15.33:0/1008254707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b4a30 0x7ff40c147020 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 --2- 172.21.15.33:0/1008254707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b9530 0x7ff40c0b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 -- 172.21.15.33:0/1008254707 >> 172.21.15.33:0/1008254707 conn(0x7ff40c072760 msgr2=0x7ff40c072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 -- 172.21.15.33:0/1008254707 shutdown_connections 2024-03-28T01:06:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.836+0000 7ff411c00640 1 -- 172.21.15.33:0/1008254707 wait complete. 2024-03-28T01:06:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.838+0000 7ff411c00640 1 Processor -- start 2024-03-28T01:06:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.838+0000 7ff411c00640 1 -- start start 2024-03-28T01:06:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff411c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b4a30 0x7ff40c13d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff411c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b9530 0x7ff40c1dcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff411c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff40c140cb0 0x7ff40c1dd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff411c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c147560 con 0x7ff40c0b9530 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff411c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0b4e90 con 0x7ff40c140cb0 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff411c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c141620 con 0x7ff40c0b4a30 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff403400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b9530 0x7ff40c1dcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff403400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b9530 0x7ff40c1dcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40290/0 (socket says 172.21.15.33:40290) 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.839+0000 7ff403400640 1 -- 172.21.15.33:0/389425986 learned_addr learned my addr 172.21.15.33:0/389425986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff411200640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff40c140cb0 0x7ff40c1dd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff40be00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b4a30 0x7ff40c13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff403400640 1 -- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b4a30 msgr2=0x7ff40c13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff403400640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b4a30 0x7ff40c13d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff403400640 1 -- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff40c140cb0 msgr2=0x7ff40c1dd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff403400640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff40c140cb0 0x7ff40c1dd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff403400640 1 -- 172.21.15.33:0/389425986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c1ce560 con 0x7ff40c0b9530 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.840+0000 7ff403400640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b9530 0x7ff40c1dcf10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4000c00 tx=0x7ff3f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.841+0000 7ff408c00640 1 -- 172.21.15.33:0/389425986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4032860 con 0x7ff40c0b9530 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.841+0000 7ff408c00640 1 -- 172.21.15.33:0/389425986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4032a20 con 0x7ff40c0b9530 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.841+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c1ce840 con 0x7ff40c0b9530 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.841+0000 7ff408c00640 1 -- 172.21.15.33:0/389425986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4032e10 con 0x7ff40c0b9530 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.841+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c1cd220 con 0x7ff40c0b9530 2024-03-28T01:06:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.842+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c075310 con 0x7ff40c0b9530 2024-03-28T01:06:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.843+0000 7ff408c00640 1 -- 172.21.15.33:0/389425986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff3f404d3f0 con 0x7ff40c0b9530 2024-03-28T01:06:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.844+0000 7ff408c00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff3e806bbd0 0x7ff3e806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.844+0000 7ff408c00640 1 -- 172.21.15.33:0/389425986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3f40ac5d0 con 0x7ff40c0b9530 2024-03-28T01:06:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.844+0000 7ff40be00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff3e806bbd0 0x7ff3e806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.845+0000 7ff40be00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff3e806bbd0 0x7ff3e806e090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b9b0 tx=0x7ff3fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:35.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:35.848+0000 7ff408c00640 1 -- 172.21.15.33:0/389425986 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff3f4040070 con 0x7ff40c0b9530 2024-03-28T01:06:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.050+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff40c0bafe0 con 0x7ff40c0b9530 2024-03-28T01:06:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.051+0000 7ff408c00640 1 -- 172.21.15.33:0/389425986 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3f4031e90 con 0x7ff40c0b9530 2024-03-28T01:06:36.053 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff3e806bbd0 msgr2=0x7ff3e806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff3e806bbd0 0x7ff3e806e090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b9b0 tx=0x7ff3fc008040 comp rx=0 tx=0).stop 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b9530 msgr2=0x7ff40c1dcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b9530 0x7ff40c1dcf10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4000c00 tx=0x7ff3f40046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 shutdown_connections 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff3e806bbd0 0x7ff3e806e090 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff40c140cb0 0x7ff40c1dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff40c0b9530 0x7ff40c1dcf10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 --2- 172.21.15.33:0/389425986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff40c0b4a30 0x7ff40c13d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 >> 172.21.15.33:0/389425986 conn(0x7ff40c072760 msgr2=0x7ff40c0bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.053+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 shutdown_connections 2024-03-28T01:06:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.054+0000 7ff411c00640 1 -- 172.21.15.33:0/389425986 wait complete. 2024-03-28T01:06:36.076 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T01:06:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.288+0000 7f2f0c800640 1 Processor -- start 2024-03-28T01:06:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.289+0000 7f2f0c800640 1 -- start start 2024-03-28T01:06:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.290+0000 7f2f0c800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f2f04067040 0x7f2f040afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.290+0000 7f2f0c800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f040afff0 0x7f2f0409b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.290+0000 7f2f0c800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f040ae260 con 0x7f2f040b5910 2024-03-28T01:06:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.290+0000 7f2f0c800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f040af620 con 0x7f2f040afff0 2024-03-28T01:06:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.290+0000 7f2f0c800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f04067420 con 0x7f2f04067040 2024-03-28T01:06:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f09400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2f040b5910 0x7f2f0409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55592/0 (socket says 172.21.15.33:55592) 2024-03-28T01:06:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f09400640 1 -- 172.21.15.33:0/4031934973 learned_addr learned my addr 172.21.15.33:0/4031934973 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f0a800640 1 --2- 172.21.15.33:0/4031934973 >> v2:172.21.15.33:3300/0 conn(0x7f2f040afff0 0x7f2f0409b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f09e00640 1 --2- 172.21.15.33:0/4031934973 >> v2:172.21.15.194:3300/0 conn(0x7f2f04067040 0x7f2f040afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f08a00640 1 -- 172.21.15.33:0/4031934973 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957943953 0 0) 0x7f2f040ae260 con 0x7f2f040b5910 2024-03-28T01:06:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f08a00640 1 -- 172.21.15.33:0/4031934973 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eec003520 con 0x7f2f040b5910 2024-03-28T01:06:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f0a800640 1 -- 172.21.15.33:0/4031934973 >> v2:172.21.15.194:3300/0 conn(0x7f2f04067040 msgr2=0x7f2f040afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f0a800640 1 --2- 172.21.15.33:0/4031934973 >> v2:172.21.15.194:3300/0 conn(0x7f2f04067040 0x7f2f040afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f0a800640 1 -- 172.21.15.33:0/4031934973 >> v1:172.21.15.33:6789/0 conn(0x7f2f040b5910 legacy=0x7f2f0409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.291+0000 7f2f0a800640 1 -- 172.21.15.33:0/4031934973 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040b5140 con 0x7f2f040afff0 2024-03-28T01:06:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.293+0000 7f2f0a800640 1 --2- 172.21.15.33:0/4031934973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0409b160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f04067600 tx=0x7f2f0002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3954bf48f2eaac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.293+0000 7f2f08a00640 1 -- 172.21.15.33:0/4031934973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00039070 con 0x7f2f040afff0 2024-03-28T01:06:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.293+0000 7f2f08a00640 1 -- 172.21.15.33:0/4031934973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0002cc60 con 0x7f2f040afff0 2024-03-28T01:06:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.293+0000 7f2f08a00640 1 -- 172.21.15.33:0/4031934973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00034640 con 0x7f2f040afff0 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.293+0000 7f2f0c800640 1 -- 172.21.15.33:0/4031934973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 msgr2=0x7f2f0409b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.293+0000 7f2f0c800640 1 --2- 172.21.15.33:0/4031934973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0409b160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f04067600 tx=0x7f2f0002beb0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.294+0000 7f2f0c800640 1 -- 172.21.15.33:0/4031934973 shutdown_connections 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.294+0000 7f2f0c800640 1 --2- 172.21.15.33:0/4031934973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0409b160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.294+0000 7f2f0c800640 1 --2- 172.21.15.33:0/4031934973 >> v2:172.21.15.194:3300/0 conn(0x7f2f04067040 0x7f2f040afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.294+0000 7f2f0c800640 1 -- 172.21.15.33:0/4031934973 >> 172.21.15.33:0/4031934973 conn(0x7f2f040b9240 msgr2=0x7f2f040b9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.294+0000 7f2f0c800640 1 -- 172.21.15.33:0/4031934973 shutdown_connections 2024-03-28T01:06:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.294+0000 7f2f0c800640 1 -- 172.21.15.33:0/4031934973 wait complete. 2024-03-28T01:06:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.295+0000 7f2f0c800640 1 Processor -- start 2024-03-28T01:06:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.295+0000 7f2f0c800640 1 -- start start 2024-03-28T01:06:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.296+0000 7f2f0c800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f04067040 0x7f2f040b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.296+0000 7f2f0c800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0413a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.296+0000 7f2f0c800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f040b5910 0x7f2f041dced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.296+0000 7f2f0c800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040b40c0 con 0x7f2f040afff0 2024-03-28T01:06:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.296+0000 7f2f0c800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040ae500 con 0x7f2f040b5910 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.296+0000 7f2f0c800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040b4240 con 0x7f2f04067040 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.296+0000 7f2f09400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0413a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0413a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40308/0 (socket says 172.21.15.33:40308) 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 -- 172.21.15.33:0/2787959961 learned_addr learned my addr 172.21.15.33:0/2787959961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09e00640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f04067040 0x7f2f040b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f0a800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f040b5910 0x7f2f041dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 -- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f04067040 msgr2=0x7f2f040b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f04067040 0x7f2f040b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 -- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f040b5910 msgr2=0x7f2f041dced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f040b5910 0x7f2f041dced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 -- 172.21.15.33:0/2787959961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f041e08d0 con 0x7f2f040afff0 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f0a800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f040b5910 0x7f2f041dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.297+0000 7f2f09400640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0413a380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c3c0 tx=0x7f2ef400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.298+0000 7f2efe000640 1 -- 172.21.15.33:0/2787959961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4017070 con 0x7f2f040afff0 2024-03-28T01:06:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.298+0000 7f2efe000640 1 -- 172.21.15.33:0/2787959961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4005ce0 con 0x7f2f040afff0 2024-03-28T01:06:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.298+0000 7f2efe000640 1 -- 172.21.15.33:0/2787959961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4007310 con 0x7f2f040afff0 2024-03-28T01:06:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.298+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f041e0bb0 con 0x7f2f040afff0 2024-03-28T01:06:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.298+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f040701e0 con 0x7f2f040afff0 2024-03-28T01:06:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.299+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f040af060 con 0x7f2f040afff0 2024-03-28T01:06:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.300+0000 7f2efe000640 1 -- 172.21.15.33:0/2787959961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2ef401a070 con 0x7f2f040afff0 2024-03-28T01:06:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.301+0000 7f2efe000640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2ee806bbd0 0x7f2ee806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.301+0000 7f2efe000640 1 -- 172.21.15.33:0/2787959961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ef4089dc0 con 0x7f2f040afff0 2024-03-28T01:06:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.301+0000 7f2f09e00640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2ee806bbd0 0x7f2ee806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.301+0000 7f2f09e00640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2ee806bbd0 0x7f2ee806e090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006fd0 tx=0x7f2ef8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.305+0000 7f2efe000640 1 -- 172.21.15.33:0/2787959961 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2f040af060 con 0x7f2f040afff0 2024-03-28T01:06:36.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.546+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f04075380 con 0x7f2f040afff0 2024-03-28T01:06:36.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.547+0000 7f2efe000640 1 -- 172.21.15.33:0/2787959961 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+3391 (secure 0 0 0) 0x7f2ef40588a0 con 0x7f2f040afff0 2024-03-28T01:06:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:36.550 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/214301400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":214301400},{"type":"v1","addr":"172.21.15.194:6833","nonce":214301400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T01:06:22.173766+0000","modified":"2024-03-28T01:06:25.239216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2290842450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2290842450},{"type":"v1","addr":"172.21.15.33:6835","nonce":2290842450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-03-28T01:06:36.550 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-03-28T01:06:36.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2ee806bbd0 msgr2=0x7f2ee806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2ee806bbd0 0x7f2ee806e090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006fd0 tx=0x7f2ef8008040 comp rx=0 tx=0).stop 2024-03-28T01:06:36.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 msgr2=0x7f2f0413a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0413a380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c3c0 tx=0x7f2ef400c880 comp rx=0 tx=0).stop 2024-03-28T01:06:36.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 shutdown_connections 2024-03-28T01:06:36.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2ee806bbd0 0x7f2ee806e090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f040b5910 0x7f2f041dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f040afff0 0x7f2f0413a380 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 --2- 172.21.15.33:0/2787959961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f04067040 0x7f2f040b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 >> 172.21.15.33:0/2787959961 conn(0x7f2f040b9240 msgr2=0x7f2f0409b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 shutdown_connections 2024-03-28T01:06:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.551+0000 7f2f0c800640 1 -- 172.21.15.33:0/2787959961 wait complete. 2024-03-28T01:06:36.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.809+0000 7f5b39a00640 1 Processor -- start 2024-03-28T01:06:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.809+0000 7f5b39a00640 1 -- start start 2024-03-28T01:06:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.811+0000 7f5b39a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b8d00 0x7f5b3414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.811+0000 7f5b39a00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b340b55b0 con 0x7f5b340b6b80 2024-03-28T01:06:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.811+0000 7f5b39a00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b340b9190 con 0x7f5b340b51d0 2024-03-28T01:06:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.811+0000 7f5b39a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3414a820 con 0x7f5b340b8d00 2024-03-28T01:06:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.811+0000 7f5b39000640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5b340b6b80 0x7f5b3414b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55602/0 (socket says 172.21.15.33:55602) 2024-03-28T01:06:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.811+0000 7f5b39000640 1 -- 172.21.15.33:0/2722848645 learned_addr learned my addr 172.21.15.33:0/2722848645 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.811+0000 7f5b2be00640 1 --2- 172.21.15.33:0/2722848645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b8d00 0x7f5b3414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.812+0000 7f5b2be00640 1 -- 172.21.15.33:0/2722848645 >> v1:172.21.15.194:6789/0 conn(0x7f5b340b51d0 legacy=0x7f5b340b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.812+0000 7f5b2be00640 1 -- 172.21.15.33:0/2722848645 >> v1:172.21.15.33:6789/0 conn(0x7f5b340b6b80 legacy=0x7f5b3414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.812+0000 7f5b2be00640 1 -- 172.21.15.33:0/2722848645 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340b9800 con 0x7f5b340b8d00 2024-03-28T01:06:36.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.813+0000 7f5b2be00640 1 --2- 172.21.15.33:0/2722848645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b8d00 0x7f5b3414ac70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0097d0 tx=0x7f5b1c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822129ac95eba263 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:36.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.814+0000 7f5b33400640 1 -- 172.21.15.33:0/2722848645 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c034070 con 0x7f5b340b8d00 2024-03-28T01:06:36.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.814+0000 7f5b33400640 1 -- 172.21.15.33:0/2722848645 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c030800 con 0x7f5b340b8d00 2024-03-28T01:06:36.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.814+0000 7f5b33400640 1 -- 172.21.15.33:0/2722848645 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c030bd0 con 0x7f5b340b8d00 2024-03-28T01:06:36.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.815+0000 7f5b39a00640 1 -- 172.21.15.33:0/2722848645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b8d00 msgr2=0x7f5b3414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.815+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2722848645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b8d00 0x7f5b3414ac70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0097d0 tx=0x7f5b1c02b940 comp rx=0 tx=0).stop 2024-03-28T01:06:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.815+0000 7f5b39a00640 1 -- 172.21.15.33:0/2722848645 shutdown_connections 2024-03-28T01:06:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.815+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2722848645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b8d00 0x7f5b3414ac70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.815+0000 7f5b39a00640 1 -- 172.21.15.33:0/2722848645 >> 172.21.15.33:0/2722848645 conn(0x7f5b34072940 msgr2=0x7f5b34072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.815+0000 7f5b39a00640 1 -- 172.21.15.33:0/2722848645 shutdown_connections 2024-03-28T01:06:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.815+0000 7f5b39a00640 1 -- 172.21.15.33:0/2722848645 wait complete. 2024-03-28T01:06:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.816+0000 7f5b39a00640 1 Processor -- start 2024-03-28T01:06:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.817+0000 7f5b39a00640 1 -- start start 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.817+0000 7f5b39a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b51d0 0x7f5b3413c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.817+0000 7f5b39a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b340b6b80 0x7f5b34138bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.817+0000 7f5b39a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b341dd430 0x7f5b34139130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.817+0000 7f5b39a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340b39c0 con 0x7f5b341dd430 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.817+0000 7f5b39a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340b44c0 con 0x7f5b340b6b80 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.817+0000 7f5b39a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34152b80 con 0x7f5b340b51d0 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b341dd430 0x7f5b34139130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b341dd430 0x7f5b34139130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40322/0 (socket says 172.21.15.33:40322) 2024-03-28T01:06:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 -- 172.21.15.33:0/2394573417 learned_addr learned my addr 172.21.15.33:0/2394573417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b33e00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b51d0 0x7f5b3413c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b2be00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b340b6b80 0x7f5b34138bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 -- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b51d0 msgr2=0x7f5b3413c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b51d0 0x7f5b3413c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 -- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b340b6b80 msgr2=0x7f5b34138bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b340b6b80 0x7f5b34138bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 -- 172.21.15.33:0/2394573417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3413f0b0 con 0x7f5b341dd430 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.818+0000 7f5b39000640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b341dd430 0x7f5b34139130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00cbf0 tx=0x7f5b2c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.819+0000 7f5b30c00640 1 -- 172.21.15.33:0/2394573417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c017070 con 0x7f5b341dd430 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.819+0000 7f5b30c00640 1 -- 172.21.15.33:0/2394573417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c00ad40 con 0x7f5b341dd430 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.819+0000 7f5b30c00640 1 -- 172.21.15.33:0/2394573417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c012600 con 0x7f5b341dd430 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.819+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b341dd8f0 con 0x7f5b341dd430 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.819+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3413b6a0 con 0x7f5b341dd430 2024-03-28T01:06:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.820+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b34075310 con 0x7f5b341dd430 2024-03-28T01:06:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.821+0000 7f5b30c00640 1 -- 172.21.15.33:0/2394573417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5b2c01a070 con 0x7f5b341dd430 2024-03-28T01:06:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.822+0000 7f5b30c00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5b1006bc00 0x7f5b1006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.822+0000 7f5b30c00640 1 -- 172.21.15.33:0/2394573417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b2c08a6a0 con 0x7f5b341dd430 2024-03-28T01:06:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.823+0000 7f5b33e00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5b1006bc00 0x7f5b1006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:36.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.823+0000 7f5b33e00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5b1006bc00 0x7f5b1006e0c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000f60 tx=0x7f5b24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:36.827+0000 7f5b30c00640 1 -- 172.21.15.33:0/2394573417 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5b2c058bd0 con 0x7f5b341dd430 2024-03-28T01:06:37.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.027+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b34149060 con 0x7f5b341dd430 2024-03-28T01:06:37.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.029+0000 7f5b30c00640 1 -- 172.21.15.33:0/2394573417 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f5b2c05cc70 con 0x7f5b341dd430 2024-03-28T01:06:37.032 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:23.710537+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5b1006bc00 msgr2=0x7f5b1006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5b1006bc00 0x7f5b1006e0c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000f60 tx=0x7f5b24009040 comp rx=0 tx=0).stop 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b341dd430 msgr2=0x7f5b34139130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b341dd430 0x7f5b34139130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00cbf0 tx=0x7f5b2c00a430 comp rx=0 tx=0).stop 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 shutdown_connections 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f5b1006bc00 0x7f5b1006e0c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b341dd430 0x7f5b34139130 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b340b6b80 0x7f5b34138bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 --2- 172.21.15.33:0/2394573417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b340b51d0 0x7f5b3413c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 >> 172.21.15.33:0/2394573417 conn(0x7f5b34072940 msgr2=0x7f5b34146780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 shutdown_connections 2024-03-28T01:06:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.032+0000 7f5b39a00640 1 -- 172.21.15.33:0/2394573417 wait complete. 2024-03-28T01:06:37.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T01:06:10.462126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T01:06:20.177809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T01:06:21.198872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T01:06:37.054 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T01:06:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.276+0000 7faf6a800640 1 Processor -- start 2024-03-28T01:06:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.276+0000 7faf6a800640 1 -- start start 2024-03-28T01:06:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.277+0000 7faf6a800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf640b6c80 0x7faf64141120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.277+0000 7faf6a800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf64141660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.277+0000 7faf6a800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf640b8ee0 con 0x7faf64072cc0 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.277+0000 7faf6a800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640b45a0 con 0x7faf640b6c80 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.277+0000 7faf6a800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640730a0 con 0x7faf640b2f70 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf69400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf64072cc0 0x7faf640b3e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55612/0 (socket says 172.21.15.33:55612) 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf69400640 1 -- 172.21.15.33:0/3804533871 learned_addr learned my addr 172.21.15.33:0/3804533871 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf68a00640 1 --2- 172.21.15.33:0/3804533871 >> v2:172.21.15.33:3300/0 conn(0x7faf640b6c80 0x7faf64141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf69e00640 1 --2- 172.21.15.33:0/3804533871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf64141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf68a00640 1 -- 172.21.15.33:0/3804533871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 msgr2=0x7faf64141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf68a00640 1 --2- 172.21.15.33:0/3804533871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf64141660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf68a00640 1 -- 172.21.15.33:0/3804533871 >> v1:172.21.15.33:6789/0 conn(0x7faf64072cc0 legacy=0x7faf640b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.278+0000 7faf68a00640 1 -- 172.21.15.33:0/3804533871 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640b9180 con 0x7faf640b6c80 2024-03-28T01:06:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.280+0000 7faf68a00640 1 --2- 172.21.15.33:0/3804533871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf640b6c80 0x7faf64141120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf500097a0 tx=0x7faf5002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a646351600c8a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.280+0000 7faf63e00640 1 -- 172.21.15.33:0/3804533871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50031780 con 0x7faf640b6c80 2024-03-28T01:06:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.280+0000 7faf63e00640 1 -- 172.21.15.33:0/3804533871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf50037070 con 0x7faf640b6c80 2024-03-28T01:06:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.280+0000 7faf63e00640 1 -- 172.21.15.33:0/3804533871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50031b40 con 0x7faf640b6c80 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 -- 172.21.15.33:0/3804533871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf640b6c80 msgr2=0x7faf64141120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 --2- 172.21.15.33:0/3804533871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf640b6c80 0x7faf64141120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf500097a0 tx=0x7faf5002b820 comp rx=0 tx=0).stop 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 -- 172.21.15.33:0/3804533871 shutdown_connections 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 --2- 172.21.15.33:0/3804533871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf64141660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 --2- 172.21.15.33:0/3804533871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf640b6c80 0x7faf64141120 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 -- 172.21.15.33:0/3804533871 >> 172.21.15.33:0/3804533871 conn(0x7faf640bce30 msgr2=0x7faf640bd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 -- 172.21.15.33:0/3804533871 shutdown_connections 2024-03-28T01:06:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.281+0000 7faf6a800640 1 -- 172.21.15.33:0/3804533871 wait complete. 2024-03-28T01:06:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.283+0000 7faf6a800640 1 Processor -- start 2024-03-28T01:06:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.283+0000 7faf6a800640 1 -- start start 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf6a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf64072cc0 0x7faf641e0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf6a800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf6413ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf6a800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf6413cf40 0x7faf641cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf6a800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b3b90 con 0x7faf64072cc0 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf6a800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b3a10 con 0x7faf6413cf40 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf6a800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b55e0 con 0x7faf640b2f70 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf69400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf64072cc0 0x7faf641e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf69400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf64072cc0 0x7faf641e0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40342/0 (socket says 172.21.15.33:40342) 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.284+0000 7faf69400640 1 -- 172.21.15.33:0/1761790157 learned_addr learned my addr 172.21.15.33:0/1761790157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf69e00640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf6413cf40 0x7faf641cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf68a00640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf6413ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf69400640 1 -- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 msgr2=0x7faf6413ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf69400640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf6413ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf69400640 1 -- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf6413cf40 msgr2=0x7faf641cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf69400640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf6413cf40 0x7faf641cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf69400640 1 -- 172.21.15.33:0/1761790157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf641cf3c0 con 0x7faf64072cc0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf68a00640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf6413ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf69400640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf64072cc0 0x7faf641e0400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c430 tx=0x7faf5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf61600640 1 -- 172.21.15.33:0/1761790157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54017070 con 0x7faf64072cc0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf61600640 1 -- 172.21.15.33:0/1761790157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54005ce0 con 0x7faf64072cc0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.285+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf641e0a90 con 0x7faf64072cc0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.286+0000 7faf61600640 1 -- 172.21.15.33:0/1761790157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54007310 con 0x7faf64072cc0 2024-03-28T01:06:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.286+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf64144f60 con 0x7faf64072cc0 2024-03-28T01:06:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.287+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf64075310 con 0x7faf64072cc0 2024-03-28T01:06:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.288+0000 7faf61600640 1 -- 172.21.15.33:0/1761790157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faf5401a070 con 0x7faf64072cc0 2024-03-28T01:06:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.289+0000 7faf61600640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7faf4406bc00 0x7faf4406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.289+0000 7faf61600640 1 -- 172.21.15.33:0/1761790157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf54089e80 con 0x7faf64072cc0 2024-03-28T01:06:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.289+0000 7faf68a00640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7faf4406bc00 0x7faf4406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.290+0000 7faf68a00640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7faf4406bc00 0x7faf4406e0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf50036000 tx=0x7faf50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.298+0000 7faf61600640 1 -- 172.21.15.33:0/1761790157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7faf54058960 con 0x7faf64072cc0 2024-03-28T01:06:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.509+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf641e0c60 con 0x7faf64072cc0 2024-03-28T01:06:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.510+0000 7faf61600640 1 -- 172.21.15.33:0/1761790157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf54058260 con 0x7faf64072cc0 2024-03-28T01:06:37.513 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7faf4406bc00 msgr2=0x7faf4406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7faf4406bc00 0x7faf4406e0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf50036000 tx=0x7faf50008030 comp rx=0 tx=0).stop 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf64072cc0 msgr2=0x7faf641e0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf64072cc0 0x7faf641e0400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c430 tx=0x7faf5400c8f0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 shutdown_connections 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7faf4406bc00 0x7faf4406e0c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf6413cf40 0x7faf641cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640b2f70 0x7faf6413ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 --2- 172.21.15.33:0/1761790157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf64072cc0 0x7faf641e0400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 >> 172.21.15.33:0/1761790157 conn(0x7faf640bce30 msgr2=0x7faf640b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 shutdown_connections 2024-03-28T01:06:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.513+0000 7faf6a800640 1 -- 172.21.15.33:0/1761790157 wait complete. 2024-03-28T01:06:37.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T01:06:37.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.751+0000 7f3e13e00640 1 Processor -- start 2024-03-28T01:06:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.751+0000 7f3e13e00640 1 -- start start 2024-03-28T01:06:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.752+0000 7f3e13e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f3e140b9530 0x7f3e140b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.752+0000 7f3e13e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e140b4a30 0x7f3e14147020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.752+0000 7f3e13e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e141410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.752+0000 7f3e13e00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140b6570 con 0x7f3e140b4a30 2024-03-28T01:06:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.752+0000 7f3e13e00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140b6300 con 0x7f3e140b9530 2024-03-28T01:06:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.752+0000 7f3e13e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140b5200 con 0x7f3e14140cb0 2024-03-28T01:06:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e140b4a30 0x7f3e14147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e140b4a30 0x7f3e14147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40356/0 (socket says 172.21.15.33:40356) 2024-03-28T01:06:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 -- 172.21.15.33:0/1244964696 learned_addr learned my addr 172.21.15.33:0/1244964696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e13400640 1 --2- 172.21.15.33:0/1244964696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e141410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e12a00640 1 --2- 172.21.15.33:0/1244964696 >> v2:172.21.15.194:3300/0 conn(0x7f3e140b9530 0x7f3e140b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 -- 172.21.15.33:0/1244964696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 msgr2=0x7f3e141410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 --2- 172.21.15.33:0/1244964696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e141410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 -- 172.21.15.33:0/1244964696 >> v2:172.21.15.194:3300/0 conn(0x7f3e140b9530 msgr2=0x7f3e140b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 --2- 172.21.15.33:0/1244964696 >> v2:172.21.15.194:3300/0 conn(0x7f3e140b9530 0x7f3e140b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.753+0000 7f3e0be00640 1 -- 172.21.15.33:0/1244964696 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140b4510 con 0x7f3e140b4a30 2024-03-28T01:06:37.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.754+0000 7f3e0be00640 1 --2- 172.21.15.33:0/1244964696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b4a30 0x7f3e14147020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b5410 tx=0x7f3dfc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8a918cb28c095f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:37.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.755+0000 7f3e12000640 1 -- 172.21.15.33:0/1244964696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0317c0 con 0x7f3e140b4a30 2024-03-28T01:06:37.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.755+0000 7f3e12000640 1 -- 172.21.15.33:0/1244964696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc037070 con 0x7f3e140b4a30 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.755+0000 7f3e12000640 1 -- 172.21.15.33:0/1244964696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc031b80 con 0x7f3e140b4a30 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.755+0000 7f3e13e00640 1 -- 172.21.15.33:0/1244964696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b4a30 msgr2=0x7f3e14147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.755+0000 7f3e13e00640 1 --2- 172.21.15.33:0/1244964696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b4a30 0x7f3e14147020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b5410 tx=0x7f3dfc02b860 comp rx=0 tx=0).stop 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.756+0000 7f3e13e00640 1 -- 172.21.15.33:0/1244964696 shutdown_connections 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.756+0000 7f3e13e00640 1 --2- 172.21.15.33:0/1244964696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e141410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.756+0000 7f3e13e00640 1 --2- 172.21.15.33:0/1244964696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b4a30 0x7f3e14147020 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.756+0000 7f3e13e00640 1 --2- 172.21.15.33:0/1244964696 >> v2:172.21.15.194:3300/0 conn(0x7f3e140b9530 0x7f3e140b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.756+0000 7f3e13e00640 1 -- 172.21.15.33:0/1244964696 >> 172.21.15.33:0/1244964696 conn(0x7f3e14072760 msgr2=0x7f3e14072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.756+0000 7f3e13e00640 1 -- 172.21.15.33:0/1244964696 shutdown_connections 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.756+0000 7f3e13e00640 1 -- 172.21.15.33:0/1244964696 wait complete. 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.758+0000 7f3e13e00640 1 Processor -- start 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.758+0000 7f3e13e00640 1 -- start start 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e13e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e140b4a30 0x7f3e1413d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e13e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b9530 0x7f3e14141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e13e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e14136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e13e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14147560 con 0x7f3e140b9530 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e13e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140b4ec0 con 0x7f3e140b4a30 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e13e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141415f0 con 0x7f3e14140cb0 2024-03-28T01:06:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e0be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b9530 0x7f3e14141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e0be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b9530 0x7f3e14141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40364/0 (socket says 172.21.15.33:40364) 2024-03-28T01:06:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.759+0000 7f3e0be00640 1 -- 172.21.15.33:0/3621663258 learned_addr learned my addr 172.21.15.33:0/3621663258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e12a00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e140b4a30 0x7f3e1413d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e13400640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e14136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0be00640 1 -- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 msgr2=0x7f3e14136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0be00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e14136c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0be00640 1 -- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e140b4a30 msgr2=0x7f3e1413d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0be00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e140b4a30 0x7f3e1413d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0be00640 1 -- 172.21.15.33:0/3621663258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14144b90 con 0x7f3e140b9530 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e12a00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e140b4a30 0x7f3e1413d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0be00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b9530 0x7f3e14141930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0aa00640 1 -- 172.21.15.33:0/3621663258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc032650 con 0x7f3e140b9530 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0aa00640 1 -- 172.21.15.33:0/3621663258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc0327f0 con 0x7f3e140b9530 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e0aa00640 1 -- 172.21.15.33:0/3621663258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc032be0 con 0x7f3e140b9530 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.760+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e14143de0 con 0x7f3e140b9530 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.761+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e14142640 con 0x7f3e140b9530 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.762+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e14075310 con 0x7f3e140b9530 2024-03-28T01:06:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.763+0000 7f3e0aa00640 1 -- 172.21.15.33:0/3621663258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3dfc04c060 con 0x7f3e140b9530 2024-03-28T01:06:37.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.764+0000 7f3e0aa00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3df006bbd0 0x7f3df006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:37.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.764+0000 7f3e0aa00640 1 -- 172.21.15.33:0/3621663258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dfc0ac060 con 0x7f3e140b9530 2024-03-28T01:06:37.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.764+0000 7f3e12a00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3df006bbd0 0x7f3df006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:37.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.765+0000 7f3e12a00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3df006bbd0 0x7f3df006e090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009ab0 tx=0x7f3e04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:37.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.768+0000 7f3e0aa00640 1 -- 172.21.15.33:0/3621663258 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3e14143de0 con 0x7f3e140b9530 2024-03-28T01:06:37.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.967+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e140b5fb0 con 0x7f3e140b9530 2024-03-28T01:06:37.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.968+0000 7f3e0aa00640 1 -- 172.21.15.33:0/3621663258 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3dfc040070 con 0x7f3e140b9530 2024-03-28T01:06:37.971 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-03-28T01:06:37.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.971+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3df006bbd0 msgr2=0x7f3df006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.971+0000 7f3e13e00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3df006bbd0 0x7f3df006e090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009ab0 tx=0x7f3e04008040 comp rx=0 tx=0).stop 2024-03-28T01:06:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.971+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b9530 msgr2=0x7f3e14141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.971+0000 7f3e13e00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b9530 0x7f3e14141930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc0046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.971+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 shutdown_connections 2024-03-28T01:06:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.971+0000 7f3e13e00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3df006bbd0 0x7f3df006e090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.972+0000 7f3e13e00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e14140cb0 0x7f3e14136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.972+0000 7f3e13e00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e140b9530 0x7f3e14141930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.972+0000 7f3e13e00640 1 --2- 172.21.15.33:0/3621663258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e140b4a30 0x7f3e1413d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.972+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 >> 172.21.15.33:0/3621663258 conn(0x7f3e14072760 msgr2=0x7f3e140bb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.972+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 shutdown_connections 2024-03-28T01:06:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:37.972+0000 7f3e13e00640 1 -- 172.21.15.33:0/3621663258 wait complete. 2024-03-28T01:06:37.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T01:06:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.207+0000 7ff719200640 1 Processor -- start 2024-03-28T01:06:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.207+0000 7ff719200640 1 -- start start 2024-03-28T01:06:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.209+0000 7ff719200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7ff714072cc0 0x7ff7140b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.209+0000 7ff719200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7140b8ee0 con 0x7ff7140b6c80 2024-03-28T01:06:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.209+0000 7ff719200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7140730a0 con 0x7ff7140b2f70 2024-03-28T01:06:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.209+0000 7ff719200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7140b9180 con 0x7ff714072cc0 2024-03-28T01:06:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.209+0000 7ff712a00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff7140b6c80 0x7ff714141120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55622/0 (socket says 172.21.15.33:55622) 2024-03-28T01:06:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.209+0000 7ff712a00640 1 -- 172.21.15.33:0/577641853 learned_addr learned my addr 172.21.15.33:0/577641853 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.209+0000 7ff713400640 1 --2- 172.21.15.33:0/577641853 >> v2:172.21.15.194:3300/0 conn(0x7ff714072cc0 0x7ff7140b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.210+0000 7ff712000640 1 -- 172.21.15.33:0/577641853 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922808791 0 0) 0x7ff7140b8ee0 con 0x7ff7140b6c80 2024-03-28T01:06:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.210+0000 7ff712000640 1 -- 172.21.15.33:0/577641853 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f8003520 con 0x7ff7140b6c80 2024-03-28T01:06:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.210+0000 7ff713400640 1 -- 172.21.15.33:0/577641853 >> v1:172.21.15.194:6789/0 conn(0x7ff7140b2f70 legacy=0x7ff714151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.210+0000 7ff712000640 1 -- 172.21.15.33:0/577641853 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1392051511 0 0) 0x7ff6f8003520 con 0x7ff7140b6c80 2024-03-28T01:06:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.210+0000 7ff713400640 1 -- 172.21.15.33:0/577641853 >> v1:172.21.15.33:6789/0 conn(0x7ff7140b6c80 legacy=0x7ff714141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.210+0000 7ff713400640 1 -- 172.21.15.33:0/577641853 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140b50c0 con 0x7ff714072cc0 2024-03-28T01:06:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.211+0000 7ff713400640 1 --2- 172.21.15.33:0/577641853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714072cc0 0x7ff7140b3e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7040097a0 tx=0x7ff70402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e16c666ba1e2a6b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.212+0000 7ff712000640 1 -- 172.21.15.33:0/577641853 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704037070 con 0x7ff714072cc0 2024-03-28T01:06:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.212+0000 7ff712000640 1 -- 172.21.15.33:0/577641853 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff704030510 con 0x7ff714072cc0 2024-03-28T01:06:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.212+0000 7ff712000640 1 -- 172.21.15.33:0/577641853 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700001670 con 0x7ff714072cc0 2024-03-28T01:06:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.212+0000 7ff719200640 1 -- 172.21.15.33:0/577641853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714072cc0 msgr2=0x7ff7140b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.212+0000 7ff719200640 1 --2- 172.21.15.33:0/577641853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714072cc0 0x7ff7140b3e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7040097a0 tx=0x7ff70402b710 comp rx=0 tx=0).stop 2024-03-28T01:06:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.213+0000 7ff719200640 1 -- 172.21.15.33:0/577641853 shutdown_connections 2024-03-28T01:06:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.213+0000 7ff719200640 1 --2- 172.21.15.33:0/577641853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714072cc0 0x7ff7140b3e90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.213+0000 7ff719200640 1 -- 172.21.15.33:0/577641853 >> 172.21.15.33:0/577641853 conn(0x7ff7140bce30 msgr2=0x7ff7140bd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.213+0000 7ff719200640 1 -- 172.21.15.33:0/577641853 shutdown_connections 2024-03-28T01:06:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.213+0000 7ff719200640 1 -- 172.21.15.33:0/577641853 wait complete. 2024-03-28T01:06:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.214+0000 7ff719200640 1 Processor -- start 2024-03-28T01:06:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.214+0000 7ff719200640 1 -- start start 2024-03-28T01:06:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.215+0000 7ff719200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff714072cc0 0x7ff7141dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.215+0000 7ff719200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7140b2f70 0x7ff71413c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.215+0000 7ff719200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7140b6c80 0x7ff71413c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.215+0000 7ff719200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140b3b90 con 0x7ff714072cc0 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.215+0000 7ff719200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140b3a10 con 0x7ff7140b2f70 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.215+0000 7ff719200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140b55e0 con 0x7ff7140b6c80 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.215+0000 7ff713400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff714072cc0 0x7ff7141dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff714072cc0 0x7ff7141dcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40374/0 (socket says 172.21.15.33:40374) 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 -- 172.21.15.33:0/2514535589 learned_addr learned my addr 172.21.15.33:0/2514535589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff712a00640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7140b2f70 0x7ff71413c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713e00640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7140b6c80 0x7ff71413c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 -- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7140b6c80 msgr2=0x7ff71413c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7140b6c80 0x7ff71413c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 -- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7140b2f70 msgr2=0x7ff71413c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7140b2f70 0x7ff71413c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 -- 172.21.15.33:0/2514535589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7141dd740 con 0x7ff714072cc0 2024-03-28T01:06:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.216+0000 7ff713400640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff714072cc0 0x7ff7141dcfb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff70402bd20 tx=0x7ff7040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.217+0000 7ff70b400640 1 -- 172.21.15.33:0/2514535589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704042070 con 0x7ff714072cc0 2024-03-28T01:06:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.217+0000 7ff70b400640 1 -- 172.21.15.33:0/2514535589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70403e870 con 0x7ff714072cc0 2024-03-28T01:06:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.217+0000 7ff70b400640 1 -- 172.21.15.33:0/2514535589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704037040 con 0x7ff714072cc0 2024-03-28T01:06:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.217+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff714137630 con 0x7ff714072cc0 2024-03-28T01:06:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.217+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71413b590 con 0x7ff714072cc0 2024-03-28T01:06:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.218+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff714075310 con 0x7ff714072cc0 2024-03-28T01:06:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.219+0000 7ff70b400640 1 -- 172.21.15.33:0/2514535589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff704045070 con 0x7ff714072cc0 2024-03-28T01:06:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.220+0000 7ff70b400640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff6f406bbd0 0x7ff6f406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.220+0000 7ff70b400640 1 -- 172.21.15.33:0/2514535589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7040ac7c0 con 0x7ff714072cc0 2024-03-28T01:06:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.220+0000 7ff712a00640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff6f406bbd0 0x7ff6f406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.221+0000 7ff712a00640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff6f406bbd0 0x7ff6f406e090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff714147dd0 tx=0x7ff700009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:38.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.225+0000 7ff70b400640 1 -- 172.21.15.33:0/2514535589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff70407b340 con 0x7ff714072cc0 2024-03-28T01:06:38.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.426+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff71413acf0 con 0x7ff714072cc0 2024-03-28T01:06:38.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.427+0000 7ff70b400640 1 -- 172.21.15.33:0/2514535589 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff70407ac40 con 0x7ff714072cc0 2024-03-28T01:06:38.429 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-03-28T01:06:38.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.429+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff6f406bbd0 msgr2=0x7ff6f406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.429+0000 7ff719200640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff6f406bbd0 0x7ff6f406e090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff714147dd0 tx=0x7ff700009040 comp rx=0 tx=0).stop 2024-03-28T01:06:38.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.429+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff714072cc0 msgr2=0x7ff7141dcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.429+0000 7ff719200640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff714072cc0 0x7ff7141dcfb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff70402bd20 tx=0x7ff7040046c0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 shutdown_connections 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff6f406bbd0 0x7ff6f406e090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7140b6c80 0x7ff71413c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7140b2f70 0x7ff71413c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 --2- 172.21.15.33:0/2514535589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff714072cc0 0x7ff7141dcfb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 >> 172.21.15.33:0/2514535589 conn(0x7ff7140bce30 msgr2=0x7ff7140b7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 shutdown_connections 2024-03-28T01:06:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.430+0000 7ff719200640 1 -- 172.21.15.33:0/2514535589 wait complete. 2024-03-28T01:06:38.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T01:06:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.666+0000 7f8bcae00640 1 Processor -- start 2024-03-28T01:06:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.666+0000 7f8bcae00640 1 -- start start 2024-03-28T01:06:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.667+0000 7f8bcae00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f8bc4072d20 0x7f8bc40b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bcae00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc4073100 con 0x7f8bc40b6ca0 2024-03-28T01:06:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bcae00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc40b5240 con 0x7f8bc41416c0 2024-03-28T01:06:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bcae00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40b47f0 con 0x7f8bc4072d20 2024-03-28T01:06:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bc9000640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8bc40b6ca0 0x7f8bc4140fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55628/0 (socket says 172.21.15.33:55628) 2024-03-28T01:06:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bc9000640 1 -- 172.21.15.33:0/1814466597 learned_addr learned my addr 172.21.15.33:0/1814466597 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bc9a00640 1 --2- 172.21.15.33:0/1814466597 >> v2:172.21.15.194:3300/0 conn(0x7f8bc4072d20 0x7f8bc40b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173419859 0 0) 0x7f8bc4073100 con 0x7f8bc40b6ca0 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.668+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bac003520 con 0x7f8bc40b6ca0 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184824992 0 0) 0x7f8bac003520 con 0x7f8bc40b6ca0 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 >> v2:172.21.15.194:3300/0 conn(0x7f8bc4072d20 msgr2=0x7f8bc40b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 --2- 172.21.15.33:0/1814466597 >> v2:172.21.15.194:3300/0 conn(0x7f8bc4072d20 0x7f8bc40b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 >> v1:172.21.15.194:6789/0 conn(0x7f8bc41416c0 legacy=0x7f8bc4151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40b5520 con 0x7f8bc40b6ca0 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f8bb0001670 con 0x7f8bc40b6ca0 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 >> v1:172.21.15.33:6789/0 conn(0x7f8bc40b6ca0 legacy=0x7f8bc4140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac0039b0 0x7f8bac003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bac004800 0x7f8bac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bac005230 0x7f8bac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40b47f0 con 0x7f8bac005230 2024-03-28T01:06:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac00c190 con 0x7f8bac0039b0 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac00c390 con 0x7f8bac004800 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bca400640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bac005230 0x7f8bac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.669+0000 7f8bc9a00640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac0039b0 0x7f8bac003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bc9000640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bac004800 0x7f8bac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bca400640 1 -- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bac004800 msgr2=0x7f8bac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bca400640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bac004800 0x7f8bac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bca400640 1 -- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac0039b0 msgr2=0x7f8bac003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bca400640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac0039b0 0x7f8bac003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bca400640 1 -- 172.21.15.33:0/1814466597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac003520 con 0x7f8bac005230 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bc9000640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bac004800 0x7f8bac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.670+0000 7f8bc9a00640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac0039b0 0x7f8bac003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.671+0000 7f8bca400640 1 -- 172.21.15.33:0/1814466597 reap_dead start 2024-03-28T01:06:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.671+0000 7f8bca400640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bac005230 0x7f8bac00b9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc003db0 tx=0x7f8bac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad23d87964257e69 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.671+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc037070 con 0x7f8bac005230 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.671+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc032600 con 0x7f8bac005230 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.671+0000 7f8bc3e00640 1 -- 172.21.15.33:0/1814466597 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc003980 con 0x7f8bac005230 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.672+0000 7f8bcae00640 1 -- 172.21.15.33:0/1814466597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bac005230 msgr2=0x7f8bac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.672+0000 7f8bcae00640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bac005230 0x7f8bac00b9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc003db0 tx=0x7f8bac005830 comp rx=0 tx=0).stop 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.672+0000 7f8bcae00640 1 -- 172.21.15.33:0/1814466597 shutdown_connections 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.672+0000 7f8bcae00640 1 --2- 172.21.15.33:0/1814466597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bac005230 0x7f8bac00b9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.672+0000 7f8bcae00640 1 -- 172.21.15.33:0/1814466597 >> 172.21.15.33:0/1814466597 conn(0x7f8bc40bce50 msgr2=0x7f8bc40bd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.672+0000 7f8bcae00640 1 -- 172.21.15.33:0/1814466597 shutdown_connections 2024-03-28T01:06:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.672+0000 7f8bcae00640 1 -- 172.21.15.33:0/1814466597 wait complete. 2024-03-28T01:06:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.674+0000 7f8bcae00640 1 Processor -- start 2024-03-28T01:06:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.674+0000 7f8bcae00640 1 -- start start 2024-03-28T01:06:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bcae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac005230 0x7f8bc4142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bcae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bc4144190 0x7f8bc41397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bcae00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4144610 0x7f8bc4139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bcae00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40b6810 con 0x7f8bc4144610 2024-03-28T01:06:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bcae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40b3b70 con 0x7f8bac005230 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bcae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41471c0 con 0x7f8bc4144190 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bca400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4144610 0x7f8bc4139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bca400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4144610 0x7f8bc4139d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40386/0 (socket says 172.21.15.33:40386) 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bca400640 1 -- 172.21.15.33:0/2324129842 learned_addr learned my addr 172.21.15.33:0/2324129842 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bc9000640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bc4144190 0x7f8bc41397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.675+0000 7f8bc9a00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac005230 0x7f8bc4142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bca400640 1 -- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bc4144190 msgr2=0x7f8bc41397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bca400640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bc4144190 0x7f8bc41397c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bca400640 1 -- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac005230 msgr2=0x7f8bc4142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bca400640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac005230 0x7f8bc4142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bca400640 1 -- 172.21.15.33:0/2324129842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc414aad0 con 0x7f8bc4144610 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bc9a00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac005230 0x7f8bc4142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bca400640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4144610 0x7f8bc4139d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc003ea0 tx=0x7f8bbc003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bc1600640 1 -- 172.21.15.33:0/2324129842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc037070 con 0x7f8bc4144610 2024-03-28T01:06:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bc1600640 1 -- 172.21.15.33:0/2324129842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc038040 con 0x7f8bc4144610 2024-03-28T01:06:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bc1600640 1 -- 172.21.15.33:0/2324129842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc003980 con 0x7f8bc4144610 2024-03-28T01:06:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.676+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc4143720 con 0x7f8bc4144610 2024-03-28T01:06:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.677+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc41df9a0 con 0x7f8bc4144610 2024-03-28T01:06:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.678+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc414a8f0 con 0x7f8bc4144610 2024-03-28T01:06:38.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.679+0000 7f8bc1600640 1 -- 172.21.15.33:0/2324129842 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8bbc031bd0 con 0x7f8bc4144610 2024-03-28T01:06:38.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.679+0000 7f8bc1600640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8ba406bbd0 0x7f8ba406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:38.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.680+0000 7f8bc1600640 1 -- 172.21.15.33:0/2324129842 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bbc0abd40 con 0x7f8bc4144610 2024-03-28T01:06:38.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.680+0000 7f8bc9a00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8ba406bbd0 0x7f8ba406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:38.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.681+0000 7f8bc9a00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8ba406bbd0 0x7f8ba406e090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8007380 tx=0x7f8bb800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.684+0000 7f8bc1600640 1 -- 172.21.15.33:0/2324129842 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8bc414a8f0 con 0x7f8bc4144610 2024-03-28T01:06:38.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.945+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bc4068a00 con 0x7f8bc4144610 2024-03-28T01:06:38.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.946+0000 7f8bc1600640 1 -- 172.21.15.33:0/2324129842 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+3391 (secure 0 0 0) 0x7f8bbc07a820 con 0x7f8bc4144610 2024-03-28T01:06:38.949 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:38.949 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/214301400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":214301400},{"type":"v1","addr":"172.21.15.194:6833","nonce":214301400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T01:06:22.173766+0000","modified":"2024-03-28T01:06:25.239216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2290842450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2290842450},{"type":"v1","addr":"172.21.15.33:6835","nonce":2290842450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-03-28T01:06:38.949 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-03-28T01:06:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8ba406bbd0 msgr2=0x7f8ba406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8ba406bbd0 0x7f8ba406e090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8007380 tx=0x7f8bb800b040 comp rx=0 tx=0).stop 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4144610 msgr2=0x7f8bc4139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4144610 0x7f8bc4139d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc003ea0 tx=0x7f8bbc003760 comp rx=0 tx=0).stop 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 shutdown_connections 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8ba406bbd0 0x7f8ba406e090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4144610 0x7f8bc4139d00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8bc4144190 0x7f8bc41397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 --2- 172.21.15.33:0/2324129842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8bac005230 0x7f8bc4142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:38.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 >> 172.21.15.33:0/2324129842 conn(0x7f8bc40bce50 msgr2=0x7f8bc40b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 shutdown_connections 2024-03-28T01:06:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:38.949+0000 7f8bcae00640 1 -- 172.21.15.33:0/2324129842 wait complete. 2024-03-28T01:06:38.971 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:06:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.182+0000 7f068b200640 1 Processor -- start 2024-03-28T01:06:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.182+0000 7f068b200640 1 -- start start 2024-03-28T01:06:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.183+0000 7f068b200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f068c0b2130 0x7f068c13edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.183+0000 7f068b200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f068c148cf0 0x7f068c14f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068b200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068c0b2510 con 0x7f068c13f330 2024-03-28T01:06:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068b200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c0b4670 con 0x7f068c148cf0 2024-03-28T01:06:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068b200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c06bef0 con 0x7f068c0b2130 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068a800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f068c148cf0 0x7f068c14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068a800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f068c148cf0 0x7f068c14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40390/0 (socket says 172.21.15.33:40390) 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068a800640 1 -- 172.21.15.33:0/1885867114 learned_addr learned my addr 172.21.15.33:0/1885867114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f0689e00640 1 --2- 172.21.15.33:0/1885867114 >> v2:172.21.15.194:3300/0 conn(0x7f068c0b2130 0x7f068c13edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068a800640 1 -- 172.21.15.33:0/1885867114 >> v2:172.21.15.194:3300/0 conn(0x7f068c0b2130 msgr2=0x7f068c13edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068a800640 1 --2- 172.21.15.33:0/1885867114 >> v2:172.21.15.194:3300/0 conn(0x7f068c0b2130 0x7f068c13edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068a800640 1 -- 172.21.15.33:0/1885867114 >> v1:172.21.15.194:6789/0 conn(0x7f068c13f330 legacy=0x7f068c1485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.184+0000 7f068a800640 1 -- 172.21.15.33:0/1885867114 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0b4c40 con 0x7f068c148cf0 2024-03-28T01:06:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.185+0000 7f068a800640 1 --2- 172.21.15.33:0/1885867114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c148cf0 0x7f068c14f850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f068c0b29a0 tx=0x7f068402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfedff4a04726139 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.186+0000 7f0688a00640 1 -- 172.21.15.33:0/1885867114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068402c960 con 0x7f068c148cf0 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.186+0000 7f0688a00640 1 -- 172.21.15.33:0/1885867114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068402cb00 con 0x7f068c148cf0 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.186+0000 7f0688a00640 1 -- 172.21.15.33:0/1885867114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06840343f0 con 0x7f068c148cf0 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.186+0000 7f068b200640 1 -- 172.21.15.33:0/1885867114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c148cf0 msgr2=0x7f068c14f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.186+0000 7f068b200640 1 --2- 172.21.15.33:0/1885867114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c148cf0 0x7f068c14f850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f068c0b29a0 tx=0x7f068402bdc0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.187+0000 7f068b200640 1 -- 172.21.15.33:0/1885867114 shutdown_connections 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.187+0000 7f068b200640 1 --2- 172.21.15.33:0/1885867114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c148cf0 0x7f068c14f850 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.187+0000 7f068b200640 1 --2- 172.21.15.33:0/1885867114 >> v2:172.21.15.194:3300/0 conn(0x7f068c0b2130 0x7f068c13edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.187+0000 7f068b200640 1 -- 172.21.15.33:0/1885867114 >> 172.21.15.33:0/1885867114 conn(0x7f068c0b89c0 msgr2=0x7f068c0b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.187+0000 7f068b200640 1 -- 172.21.15.33:0/1885867114 shutdown_connections 2024-03-28T01:06:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.187+0000 7f068b200640 1 -- 172.21.15.33:0/1885867114 wait complete. 2024-03-28T01:06:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.188+0000 7f068b200640 1 Processor -- start 2024-03-28T01:06:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.188+0000 7f068b200640 1 -- start start 2024-03-28T01:06:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f068b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c0b2130 0x7f068c0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f068b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f068c13f330 0x7f068c13bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f068b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f068c148cf0 0x7f068c1e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f068b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c072d20 con 0x7f068c0b2130 2024-03-28T01:06:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f068b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0b4ec0 con 0x7f068c13f330 2024-03-28T01:06:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f068b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0b0870 con 0x7f068c148cf0 2024-03-28T01:06:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f0689e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c0b2130 0x7f068c0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f0689e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c0b2130 0x7f068c0b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40406/0 (socket says 172.21.15.33:40406) 2024-03-28T01:06:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.189+0000 7f0689e00640 1 -- 172.21.15.33:0/2151242187 learned_addr learned my addr 172.21.15.33:0/2151242187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f0689400640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f068c13f330 0x7f068c13bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f068a800640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f068c148cf0 0x7f068c1e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f0689e00640 1 -- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f068c148cf0 msgr2=0x7f068c1e0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f0689e00640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f068c148cf0 0x7f068c1e0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f0689e00640 1 -- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f068c13f330 msgr2=0x7f068c13bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f0689e00640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f068c13f330 0x7f068c13bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f0689e00640 1 -- 172.21.15.33:0/2151242187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0b0220 con 0x7f068c0b2130 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f068a800640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f068c148cf0 0x7f068c1e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.190+0000 7f0689e00640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c0b2130 0x7f068c0b1110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f067c00d5f0 tx=0x7f067c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.191+0000 7f0682000640 1 -- 172.21.15.33:0/2151242187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0125f0 con 0x7f068c0b2130 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.191+0000 7f0682000640 1 -- 172.21.15.33:0/2151242187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c019070 con 0x7f068c0b2130 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.191+0000 7f0682000640 1 -- 172.21.15.33:0/2151242187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0129d0 con 0x7f068c0b2130 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.191+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c0b0500 con 0x7f068c0b2130 2024-03-28T01:06:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.191+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c145c70 con 0x7f068c0b2130 2024-03-28T01:06:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.192+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c075310 con 0x7f068c0b2130 2024-03-28T01:06:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.193+0000 7f0682000640 1 -- 172.21.15.33:0/2151242187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f067c012b70 con 0x7f068c0b2130 2024-03-28T01:06:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.194+0000 7f0682000640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f066c06bbd0 0x7f066c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.194+0000 7f0682000640 1 -- 172.21.15.33:0/2151242187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f067c08ae10 con 0x7f068c0b2130 2024-03-28T01:06:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.194+0000 7f0689400640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f066c06bbd0 0x7f066c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.195+0000 7f0689400640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f066c06bbd0 0x7f066c06e090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.198+0000 7f0682000640 1 -- 172.21.15.33:0/2151242187 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f067c08e030 con 0x7f068c0b2130 2024-03-28T01:06:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.469+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f068c06b470 con 0x7f068c0b2130 2024-03-28T01:06:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.472+0000 7f0682000640 1 -- 172.21.15.33:0/2151242187 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f067c0598f0 con 0x7f068c0b2130 2024-03-28T01:06:39.475 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:39.475 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:06:23.710537+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f066c06bbd0 msgr2=0x7f066c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f066c06bbd0 0x7f066c06e090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678008040 comp rx=0 tx=0).stop 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c0b2130 msgr2=0x7f068c0b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c0b2130 0x7f068c0b1110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f067c00d5f0 tx=0x7f067c00dab0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 shutdown_connections 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f066c06bbd0 0x7f066c06e090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f068c148cf0 0x7f068c1e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f068c13f330 0x7f068c13bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 --2- 172.21.15.33:0/2151242187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c0b2130 0x7f068c0b1110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 >> 172.21.15.33:0/2151242187 conn(0x7f068c0b89c0 msgr2=0x7f068c13d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 shutdown_connections 2024-03-28T01:06:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.474+0000 7f068b200640 1 -- 172.21.15.33:0/2151242187 wait complete. 2024-03-28T01:06:39.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T01:06:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.709+0000 7f13ffe00640 1 Processor -- start 2024-03-28T01:06:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.709+0000 7f13ffe00640 1 -- start start 2024-03-28T01:06:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ffe00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f14000b51d0 0x7f14000b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ffe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b8d00 0x7f140014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ffe00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14000b42f0 0x7f140014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ffe00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140014a820 con 0x7f14000b42f0 2024-03-28T01:06:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ffe00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14000b55b0 con 0x7f14000b51d0 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ffe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b9190 con 0x7f14000b8d00 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ff400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14000b42f0 0x7f140014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ff400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14000b42f0 0x7f140014b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40410/0 (socket says 172.21.15.33:40410) 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13ff400640 1 -- 172.21.15.33:0/2750627536 learned_addr learned my addr 172.21.15.33:0/2750627536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13fea00640 1 --2- 172.21.15.33:0/2750627536 >> v2:172.21.15.194:3300/0 conn(0x7f14000b51d0 0x7f14000b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.711+0000 7f13f7e00640 1 --2- 172.21.15.33:0/2750627536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b8d00 0x7f140014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.712+0000 7f13ff400640 1 -- 172.21.15.33:0/2750627536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b8d00 msgr2=0x7f140014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.712+0000 7f13ff400640 1 --2- 172.21.15.33:0/2750627536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b8d00 0x7f140014ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.712+0000 7f13ff400640 1 -- 172.21.15.33:0/2750627536 >> v2:172.21.15.194:3300/0 conn(0x7f14000b51d0 msgr2=0x7f14000b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.712+0000 7f13ff400640 1 --2- 172.21.15.33:0/2750627536 >> v2:172.21.15.194:3300/0 conn(0x7f14000b51d0 0x7f14000b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.712+0000 7f13ff400640 1 -- 172.21.15.33:0/2750627536 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000b9800 con 0x7f14000b42f0 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.713+0000 7f13ff400640 1 --2- 172.21.15.33:0/2750627536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14000b42f0 0x7f140014b1b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14000ba2b0 tx=0x7f13f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28bb5fffa874fec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.713+0000 7f13fe000640 1 -- 172.21.15.33:0/2750627536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f802ca80 con 0x7f14000b42f0 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.713+0000 7f13fe000640 1 -- 172.21.15.33:0/2750627536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f802cc20 con 0x7f14000b42f0 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.713+0000 7f13fe000640 1 -- 172.21.15.33:0/2750627536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80343f0 con 0x7f14000b42f0 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 -- 172.21.15.33:0/2750627536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14000b42f0 msgr2=0x7f140014b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2750627536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14000b42f0 0x7f140014b1b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14000ba2b0 tx=0x7f13f802bee0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 -- 172.21.15.33:0/2750627536 shutdown_connections 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2750627536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14000b42f0 0x7f140014b1b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2750627536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b8d00 0x7f140014ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2750627536 >> v2:172.21.15.194:3300/0 conn(0x7f14000b51d0 0x7f14000b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 -- 172.21.15.33:0/2750627536 >> 172.21.15.33:0/2750627536 conn(0x7f1400072940 msgr2=0x7f1400072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:39.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 -- 172.21.15.33:0/2750627536 shutdown_connections 2024-03-28T01:06:39.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.714+0000 7f13ffe00640 1 -- 172.21.15.33:0/2750627536 wait complete. 2024-03-28T01:06:39.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.715+0000 7f13ffe00640 1 Processor -- start 2024-03-28T01:06:39.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.716+0000 7f13ffe00640 1 -- start start 2024-03-28T01:06:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.716+0000 7f13ffe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b42f0 0x7f14000bc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.716+0000 7f13ffe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14000b51d0 0x7f14001d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.716+0000 7f13ffe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14001dca30 0x7f14001dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.716+0000 7f13ffe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b39c0 con 0x7f14001dca30 2024-03-28T01:06:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.716+0000 7f13ffe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140014b6f0 con 0x7f14000b51d0 2024-03-28T01:06:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.716+0000 7f13ffe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b6b80 con 0x7f14000b42f0 2024-03-28T01:06:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14001dca30 0x7f14001dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14001dca30 0x7f14001dcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40420/0 (socket says 172.21.15.33:40420) 2024-03-28T01:06:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 -- 172.21.15.33:0/2719040329 learned_addr learned my addr 172.21.15.33:0/2719040329 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13fea00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b42f0 0x7f14000bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13f7e00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14000b51d0 0x7f14001d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 -- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b42f0 msgr2=0x7f14000bc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b42f0 0x7f14000bc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 -- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14000b51d0 msgr2=0x7f14001d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14000b51d0 0x7f14001d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13ff400640 1 -- 172.21.15.33:0/2719040329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14001d0df0 con 0x7f14001dca30 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.717+0000 7f13fea00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b42f0 0x7f14000bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.718+0000 7f13ff400640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14001dca30 0x7f14001dcee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13f8036040 tx=0x7f13f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.718+0000 7f13f6a00640 1 -- 172.21.15.33:0/2719040329 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8043020 con 0x7f14001dca30 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.718+0000 7f13f6a00640 1 -- 172.21.15.33:0/2719040329 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f803e450 con 0x7f14001dca30 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.718+0000 7f13f6a00640 1 -- 172.21.15.33:0/2719040329 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8009510 con 0x7f14001dca30 2024-03-28T01:06:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.718+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14001d1070 con 0x7f14001dca30 2024-03-28T01:06:39.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.718+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1400144990 con 0x7f14001dca30 2024-03-28T01:06:39.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.719+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1400144cc0 con 0x7f14001dca30 2024-03-28T01:06:39.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.720+0000 7f13f6a00640 1 -- 172.21.15.33:0/2719040329 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f13f8046020 con 0x7f14001dca30 2024-03-28T01:06:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.721+0000 7f13f6a00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f13dc06bbd0 0x7f13dc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.721+0000 7f13f6a00640 1 -- 172.21.15.33:0/2719040329 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13f80ad1f0 con 0x7f14001dca30 2024-03-28T01:06:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.722+0000 7f13fea00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f13dc06bbd0 0x7f13dc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.722+0000 7f13fea00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f13dc06bbd0 0x7f13dc06e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13f0006fd0 tx=0x7f13f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.730+0000 7f13f6a00640 1 -- 172.21.15.33:0/2719040329 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f13f804d030 con 0x7f14001dca30 2024-03-28T01:06:39.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.975+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1400144eb0 con 0x7f14001dca30 2024-03-28T01:06:39.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.976+0000 7f13f6a00640 1 -- 172.21.15.33:0/2719040329 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+3391 (secure 0 0 0) 0x7f14001d0df0 con 0x7f14001dca30 2024-03-28T01:06:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:06:39.979 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/214301400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":214301400},{"type":"v1","addr":"172.21.15.194:6833","nonce":214301400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T01:06:22.173766+0000","modified":"2024-03-28T01:06:25.239216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2290842450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2290842450},{"type":"v1","addr":"172.21.15.33:6835","nonce":2290842450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-03-28T01:06:39.979 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-03-28T01:06:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f13dc06bbd0 msgr2=0x7f13dc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f13dc06bbd0 0x7f13dc06e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13f0006fd0 tx=0x7f13f0008040 comp rx=0 tx=0).stop 2024-03-28T01:06:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14001dca30 msgr2=0x7f14001dcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14001dca30 0x7f14001dcee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13f8036040 tx=0x7f13f8004a40 comp rx=0 tx=0).stop 2024-03-28T01:06:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 shutdown_connections 2024-03-28T01:06:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f13dc06bbd0 0x7f13dc06e090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14001dca30 0x7f14001dcee0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.979+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14000b51d0 0x7f14001d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.980+0000 7f13ffe00640 1 --2- 172.21.15.33:0/2719040329 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14000b42f0 0x7f14000bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:39.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.980+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 >> 172.21.15.33:0/2719040329 conn(0x7f1400072940 msgr2=0x7f1400148690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:39.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.980+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 shutdown_connections 2024-03-28T01:06:39.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:39.980+0000 7f13ffe00640 1 -- 172.21.15.33:0/2719040329 wait complete. 2024-03-28T01:06:40.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T01:06:22.173766+0000', 'modified': '2024-03-28T01:06:25.239216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2290842450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 2290842450}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2290842450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4245, 'qdb_cluster': [4245]} 2024-03-28T01:06:40.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T01:06:40.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-28T01:06:40.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.460+0000 7f0c12800640 1 Processor -- start 2024-03-28T01:06:40.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.460+0000 7f0c12800640 1 -- start start 2024-03-28T01:06:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c12800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c0c0b5550 0x7f0c0c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c12800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0727d0 0x7f0c0c141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c12800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c0b0e70 con 0x7f0c0c141580 2024-03-28T01:06:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c12800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c141a00 con 0x7f0c0c0b5550 2024-03-28T01:06:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c12800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0b5930 con 0x7f0c0c0727d0 2024-03-28T01:06:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c11400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c0c0b5550 0x7f0c0c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c11e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0c0c141580 0x7f0c0c14a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55636/0 (socket says 172.21.15.33:55636) 2024-03-28T01:06:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c11e00640 1 -- 172.21.15.33:0/1876976872 learned_addr learned my addr 172.21.15.33:0/1876976872 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:06:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.462+0000 7f0c10a00640 1 --2- 172.21.15.33:0/1876976872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0727d0 0x7f0c0c141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.463+0000 7f0c0be00640 1 -- 172.21.15.33:0/1876976872 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061964550 0 0) 0x7f0c0c0b0e70 con 0x7f0c0c141580 2024-03-28T01:06:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.463+0000 7f0c0be00640 1 -- 172.21.15.33:0/1876976872 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf4003520 con 0x7f0c0c141580 2024-03-28T01:06:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.463+0000 7f0c11400640 1 -- 172.21.15.33:0/1876976872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0727d0 msgr2=0x7f0c0c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.463+0000 7f0c11400640 1 --2- 172.21.15.33:0/1876976872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0727d0 0x7f0c0c141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.463+0000 7f0c11400640 1 -- 172.21.15.33:0/1876976872 >> v1:172.21.15.33:6789/0 conn(0x7f0c0c141580 legacy=0x7f0c0c14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.463+0000 7f0c11400640 1 -- 172.21.15.33:0/1876976872 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0b1110 con 0x7f0c0c0b5550 2024-03-28T01:06:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.463+0000 7f0c10a00640 1 --2- 172.21.15.33:0/1876976872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0727d0 0x7f0c0c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:06:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.464+0000 7f0c11400640 1 --2- 172.21.15.33:0/1876976872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0b5550 0x7f0c0c071a80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6c0091858360dc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.465+0000 7f0c0be00640 1 -- 172.21.15.33:0/1876976872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00031780 con 0x7f0c0c0b5550 2024-03-28T01:06:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.465+0000 7f0c0be00640 1 -- 172.21.15.33:0/1876976872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c00037070 con 0x7f0c0c0b5550 2024-03-28T01:06:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.465+0000 7f0c0be00640 1 -- 172.21.15.33:0/1876976872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00031b20 con 0x7f0c0c0b5550 2024-03-28T01:06:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.465+0000 7f0c12800640 1 -- 172.21.15.33:0/1876976872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0b5550 msgr2=0x7f0c0c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.465+0000 7f0c12800640 1 --2- 172.21.15.33:0/1876976872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0b5550 0x7f0c0c071a80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b820 comp rx=0 tx=0).stop 2024-03-28T01:06:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.465+0000 7f0c12800640 1 -- 172.21.15.33:0/1876976872 shutdown_connections 2024-03-28T01:06:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.465+0000 7f0c12800640 1 --2- 172.21.15.33:0/1876976872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0727d0 0x7f0c0c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.466+0000 7f0c12800640 1 --2- 172.21.15.33:0/1876976872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0b5550 0x7f0c0c071a80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.466+0000 7f0c12800640 1 -- 172.21.15.33:0/1876976872 >> 172.21.15.33:0/1876976872 conn(0x7f0c0c0b63d0 msgr2=0x7f0c0c0b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.466+0000 7f0c12800640 1 -- 172.21.15.33:0/1876976872 shutdown_connections 2024-03-28T01:06:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.466+0000 7f0c12800640 1 -- 172.21.15.33:0/1876976872 wait complete. 2024-03-28T01:06:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.467+0000 7f0c12800640 1 Processor -- start 2024-03-28T01:06:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.467+0000 7f0c12800640 1 -- start start 2024-03-28T01:06:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.468+0000 7f0c12800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c0c0727d0 0x7f0c0c0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.468+0000 7f0c12800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0b5550 0x7f0c0c1dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.468+0000 7f0c12800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c141580 0x7f0c0c1dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.468+0000 7f0c12800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c14af20 con 0x7f0c0c141580 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.468+0000 7f0c12800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c071fc0 con 0x7f0c0c0727d0 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.468+0000 7f0c12800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c14b0a0 con 0x7f0c0c0b5550 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.468+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c141580 0x7f0c0c1dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c141580 0x7f0c0c1dd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40438/0 (socket says 172.21.15.33:40438) 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 -- 172.21.15.33:0/3407098148 learned_addr learned my addr 172.21.15.33:0/3407098148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 -- 172.21.15.33:0/3407098148 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0b5550 msgr2=0x7f0c0c1dcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0b5550 0x7f0c0c1dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 -- 172.21.15.33:0/3407098148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c0c0727d0 msgr2=0x7f0c0c0afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c0c0727d0 0x7f0c0c0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 -- 172.21.15.33:0/3407098148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c142a60 con 0x7f0c0c141580 2024-03-28T01:06:40.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c11e00640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c141580 0x7f0c0c1dd4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400caa0 tx=0x7f0c0400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:40.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c09600640 1 -- 172.21.15.33:0/3407098148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04017070 con 0x7f0c0c141580 2024-03-28T01:06:40.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.469+0000 7f0c09600640 1 -- 172.21.15.33:0/3407098148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0400ac00 con 0x7f0c0c141580 2024-03-28T01:06:40.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.470+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c142d40 con 0x7f0c0c141580 2024-03-28T01:06:40.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.470+0000 7f0c09600640 1 -- 172.21.15.33:0/3407098148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04012400 con 0x7f0c0c141580 2024-03-28T01:06:40.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.470+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c1480a0 con 0x7f0c0c141580 2024-03-28T01:06:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.473+0000 7f0c09600640 1 -- 172.21.15.33:0/3407098148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0c0401a070 con 0x7f0c0c141580 2024-03-28T01:06:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.473+0000 7f0c16b99740 1 Processor -- start 2024-03-28T01:06:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.473+0000 7f0c16b99740 1 -- start start 2024-03-28T01:06:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c09600640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f0bec06bbd0 0x7f0bec06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c09600640 1 -- 172.21.15.33:0/3407098148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c0408a770 con 0x7f0c0c141580 2024-03-28T01:06:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb5b7a0 0x55597eb83f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c10a00640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f0bec06bbd0 0x7f0bec06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb8aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c16b99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597eac0c90 con 0x55597eb84480 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597ea69980 con 0x55597eb5b7a0 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.474+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x55597eabcd30 con 0x55597eb7fe10 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb7f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40446/0 (socket says 172.21.15.33:40446) 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 -- 172.21.15.33:0/26893581 learned_addr learned my addr 172.21.15.33:0/26893581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c11400640 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb5b7a0 0x55597eb83f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c11e00640 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb8aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 -- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 msgr2=0x55597eb8aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb8aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 -- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb5b7a0 msgr2=0x55597eb83f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb5b7a0 0x55597eb83f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 -- 172.21.15.33:0/26893581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55597ea81510 con 0x55597eb84480 2024-03-28T01:06:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.475+0000 7f0c10a00640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f0bec06bbd0 0x7f0bec06e090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8006f00 tx=0x7f0bf8015670 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.476+0000 7f0c10a00640 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb7f8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b5c30 tx=0x7f0bf801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce1eeff169c440c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.476+0000 7f0bfdc00640 1 -- 172.21.15.33:0/26893581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8022410 con 0x55597eb84480 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.476+0000 7f0bfdc00640 1 -- 172.21.15.33:0/26893581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80225b0 con 0x55597eb84480 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.476+0000 7f0bfdc00640 1 -- 172.21.15.33:0/26893581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8022980 con 0x55597eb84480 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.476+0000 7f0c16b99740 1 -- 172.21.15.33:0/26893581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 msgr2=0x55597eb7f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.476+0000 7f0c16b99740 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb7f8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b5c30 tx=0x7f0bf801c000 comp rx=0 tx=0).stop 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 -- 172.21.15.33:0/26893581 shutdown_connections 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb8aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb7f8d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 --2- 172.21.15.33:0/26893581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb5b7a0 0x55597eb83f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 -- 172.21.15.33:0/26893581 >> 172.21.15.33:0/26893581 conn(0x55597eb58d70 msgr2=0x55597eb5acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 -- 172.21.15.33:0/26893581 shutdown_connections 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 -- 172.21.15.33:0/26893581 wait complete. 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 -- start start 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.477+0000 7f0c16b99740 1 Processor -- start 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.478+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb86ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.478+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb82aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.478+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7e390 0x55597eb82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.478+0000 7f0c16b99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597eb84900 con 0x55597eb84480 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.478+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597ea7cdd0 con 0x55597eb7e390 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.478+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x55597eb8a510 con 0x55597eb7fe10 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.478+0000 7f0c16b99740 10 client.0 _ll_register_callbacks cb 0x55597e7d4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T01:06:40.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb82aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb82aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40448/0 (socket says 172.21.15.33:40448) 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 -- 172.21.15.33:0/2916301507 learned_addr learned my addr 172.21.15.33:0/2916301507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c11400640 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb86ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c11e00640 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7e390 0x55597eb82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 -- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 msgr2=0x55597eb86ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb86ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 -- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7e390 msgr2=0x55597eb82fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7e390 0x55597eb82fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 -- 172.21.15.33:0/2916301507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55597e7d52d0 con 0x55597eb84480 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0c10a00640 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb82aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8005d00 tx=0x7f0bf8006df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dc8b91552e35ed server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.479+0000 7f0bf2000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:06:40.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0bf2000640 1 -- 172.21.15.33:0/2916301507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8032020 con 0x55597eb84480 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0bf2000640 1 -- 172.21.15.33:0/2916301507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bf8022960 con 0x55597eb84480 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf80225b0 con 0x55597eb84480 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0bf2000640 1 -- 172.21.15.33:0/2916301507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf803c050 con 0x55597eb84480 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 10 client.4617 fetch_fsmap learned FSMap version 8 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55597e89f6b0 con 0x55597eb84480 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0bf2000640 1 -- 172.21.15.33:0/2916301507 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80225b0 con 0x55597eb84480 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0bf2000640 1 -- 172.21.15.33:0/2916301507 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 8) v1 ==== 2164+0+0 (secure 0 0 0) 0x7f0bf803f020 con 0x55597eb84480 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 10 client.4617 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 10 client.4617 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 20 client.4617 populate_metadata read hostname 'smithi033' 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597eb85790 0x55597eb85b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 4 client.4617 mds_command: new command op to 4245 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.480+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55597ea62520 con 0x55597eb85790 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.481+0000 7f0c11400640 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597eb85790 0x55597eb85b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.481+0000 7f0c11400640 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597eb85790 0x55597eb85b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ecb03b229cdae8 server_cookie=4567d211b082fb59 in_seq=0 out_seq=0 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.481+0000 7f0bf2000640 10 client.4617 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0bf2000640 1 -- 172.21.15.33:0/2916301507 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55597ea62520 con 0x55597eb85790 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0bf2000640 10 client.4617 handle_command_reply: tid=0 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 client.4617 shutdown 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 msgr2=0x55597eb82aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb82aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8005d00 tx=0x7f0bf8006df0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 shutdown_connections 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597eb85790 0x55597eb85b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7e390 0x55597eb82fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb82aa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 --2- 172.21.15.33:0/2916301507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7fe10 0x55597eb86ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 >> 172.21.15.33:0/2916301507 conn(0x55597eb5b7a0 msgr2=0x55597eb55c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.482+0000 7f0bf2000640 0 client.4617 ms_handle_reset on v2:172.21.15.33:6834/2290842450 2024-03-28T01:06:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.483+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 shutdown_connections 2024-03-28T01:06:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.483+0000 7f0c16b99740 1 -- 172.21.15.33:0/2916301507 wait complete. 2024-03-28T01:06:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.483+0000 7f0c16b99740 20 client.4617 trim_cache size 0 max 16384 2024-03-28T01:06:40.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 Processor -- start 2024-03-28T01:06:40.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 -- start start 2024-03-28T01:06:40.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eb5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597eb55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c1e91a0 con 0x55597eb84480 2024-03-28T01:06:40.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c13c550 con 0x55597eb7fe10 2024-03-28T01:06:40.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.501+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x55597eb87b10 con 0x55597eb7e390 2024-03-28T01:06:40.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb826f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40458/0 (socket says 172.21.15.33:40458) 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 -- 172.21.15.33:0/1176348563 learned_addr learned my addr 172.21.15.33:0/1176348563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11400640 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eb5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c10a00640 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597eb55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 -- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 msgr2=0x55597eb5bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eb5bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 -- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 msgr2=0x55597eb55fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597eb55fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 -- 172.21.15.33:0/1176348563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55597eaf5bf0 con 0x55597eb84480 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c10a00640 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597eb55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.502+0000 7f0c11e00640 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb826f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400ca70 tx=0x7f0c04054fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9b12b82b684030 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:40.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0bfd200640 1 -- 172.21.15.33:0/1176348563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04087070 con 0x55597eb84480 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0bfd200640 1 -- 172.21.15.33:0/1176348563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04061300 con 0x55597eb84480 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0bfd200640 1 -- 172.21.15.33:0/1176348563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04070070 con 0x55597eb84480 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 -- 172.21.15.33:0/1176348563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 msgr2=0x55597eb826f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb826f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400ca70 tx=0x7f0c04054fc0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 -- 172.21.15.33:0/1176348563 shutdown_connections 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597eb826f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597eb55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 --2- 172.21.15.33:0/1176348563 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eb5bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 -- 172.21.15.33:0/1176348563 >> 172.21.15.33:0/1176348563 conn(0x55597eb5b7a0 msgr2=0x55597eb88de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 -- 172.21.15.33:0/1176348563 shutdown_connections 2024-03-28T01:06:40.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.503+0000 7f0c16b99740 1 -- 172.21.15.33:0/1176348563 wait complete. 2024-03-28T01:06:40.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.504+0000 7f0c16b99740 1 -- start start 2024-03-28T01:06:40.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.504+0000 7f0c16b99740 1 Processor -- start 2024-03-28T01:06:40.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eba0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597ebf3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c16b99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597ebf41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c16b99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597eb83520 con 0x55597eb84480 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597ebfa3b0 con 0x55597eb7fe10 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c16b99740 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x55597eba0620 con 0x55597eb7e390 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c16b99740 10 client.0 _ll_register_callbacks cb 0x7f0bf40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597ebf41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597ebf41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40470/0 (socket says 172.21.15.33:40470) 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.505+0000 7f0c11e00640 1 -- 172.21.15.33:0/865921513 learned_addr learned my addr 172.21.15.33:0/865921513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11400640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eba0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c10a00640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597ebf3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11e00640 1 -- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 msgr2=0x55597eba0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11e00640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eba0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11e00640 1 -- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 msgr2=0x55597ebf3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11e00640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597ebf3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11e00640 1 -- 172.21.15.33:0/865921513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55597ebf9ee0 con 0x55597eb84480 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11400640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eba0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c10a00640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597ebf3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c11e00640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597ebf41f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400cfd0 tx=0x7f0c04001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2464924dfc28c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0bf2000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0bf2000640 1 -- 172.21.15.33:0/865921513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04087070 con 0x55597eb84480 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0bf2000640 1 -- 172.21.15.33:0/865921513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c04061070 con 0x55597eb84480 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c04061300 con 0x55597eb84480 2024-03-28T01:06:40.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0bf2000640 1 -- 172.21.15.33:0/865921513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c04070040 con 0x55597eb84480 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c16b99740 10 client.4623 fetch_fsmap learned FSMap version 8 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.506+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55597ebfa1b0 con 0x55597eb84480 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0bf2000640 1 -- 172.21.15.33:0/865921513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04077e50 con 0x55597eb84480 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0bf2000640 1 -- 172.21.15.33:0/865921513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 8) v1 ==== 2164+0+0 (secure 0 0 0) 0x7f0c04004b60 con 0x55597eb84480 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c16b99740 10 client.4623 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c16b99740 10 client.4623 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c16b99740 20 client.4623 populate_metadata read hostname 'smithi033' 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c16b99740 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597ebad9c0 0x55597ebadda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c16b99740 4 client.4623 mds_command: new command op to 4245 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55597eace1d0 con 0x55597ebad9c0 2024-03-28T01:06:40.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c11400640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597ebad9c0 0x55597ebadda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0c11400640 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597ebad9c0 0x55597ebadda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871c82f925459d1c server_cookie=7e9a3b3d84e76262 in_seq=0 out_seq=0 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.507+0000 7f0bf2000640 10 client.4623 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0bf2000640 1 -- 172.21.15.33:0/865921513 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55597eace1d0 con 0x55597ebad9c0 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0bf2000640 10 client.4623 handle_command_reply: tid=0 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 client.4623 shutdown 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 msgr2=0x55597ebf41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597ebf41f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400cfd0 tx=0x7f0c04001250 comp rx=0 tx=0).stop 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 shutdown_connections 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x55597ebad9c0 0x55597ebadda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55597eb84480 0x55597ebf41f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55597eb7fe10 0x55597ebf3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 --2- 172.21.15.33:0/865921513 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55597eb7e390 0x55597eba0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0bf2000640 0 client.4623 ms_handle_reset on v2:172.21.15.33:6834/2290842450 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 >> 172.21.15.33:0/865921513 conn(0x55597eb5b7a0 msgr2=0x55597eb802f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 shutdown_connections 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 1 -- 172.21.15.33:0/865921513 wait complete. 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.508+0000 7f0c16b99740 20 client.4623 trim_cache size 0 max 16384 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "6560509b-7594-459b-89e1-d55d3bb3e1bc", 2024-03-28T01:06:40.517 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4245, 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 17.812077395999999, 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 8, 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 20, 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 20, 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 23.431078345 2024-03-28T01:06:40.518 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:06:40.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f0bec06bbd0 msgr2=0x7f0bec06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f0bec06bbd0 0x7f0bec06e090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8006f00 tx=0x7f0bf8015670 comp rx=0 tx=0).stop 2024-03-28T01:06:40.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c141580 msgr2=0x7f0c0c1dd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:06:40.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c141580 0x7f0c0c1dd4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400caa0 tx=0x7f0c0400cf60 comp rx=0 tx=0).stop 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 shutdown_connections 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f0bec06bbd0 0x7f0bec06e090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c141580 0x7f0c0c1dd4b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c0c0b5550 0x7f0c0c1dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 --2- 172.21.15.33:0/3407098148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c0c0727d0 0x7f0c0c0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 >> 172.21.15.33:0/3407098148 conn(0x7f0c0c0b63d0 msgr2=0x7f0c0c0b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 shutdown_connections 2024-03-28T01:06:40.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:06:40.509+0000 7f0c12800640 1 -- 172.21.15.33:0/3407098148 wait complete. 2024-03-28T01:06:40.530 INFO:teuthology.run_tasks:Running task workunit... 2024-03-28T01:06:40.548 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T01:06:40.548 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-28T01:06:40.548 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-28T01:06:40.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T01:06:40.602 INFO:teuthology.orchestra.run.smithi033.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-28T01:06:40.602 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-28T01:06:40.647 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-28T01:06:40.647 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-28T01:06:40.743 INFO:tasks.workunit:timeout=3h 2024-03-28T01:06:40.743 INFO:tasks.workunit:cleanup=True 2024-03-28T01:06:40.744 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T01:06:40.797 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-28T01:06:44.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:06:44.282 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:07:14.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:07:14.434 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:07:44.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:07:44.576 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:08:11.509 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 55% (6753/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 86% (10571/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-28T01:08:12.129 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-28T01:08:12.129 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-03-28T01:08:12.129 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-28T01:08:12.129 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-03-28T01:08:12.130 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-28T01:08:12.138 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-28T01:08:12.204 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-28T01:08:12.213 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-28T01:08:12.213 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-28T01:08:12.940 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-28T01:08:13.058 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-28T01:08:13.141 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-28T01:08:13.147 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-28T01:08:13.147 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-28T01:08:13.227 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-28T01:08:13.237 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-03-28T01:08:13.237 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-28T01:08:13.288 INFO:tasks.workunit:Running workunits matching client/test.sh on client.0... 2024-03-28T01:08:13.290 INFO:tasks.workunit:Running workunit client/test.sh... 2024-03-28T01:08:13.290 DEBUG:teuthology.orchestra.run.smithi033:workunit test client/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/client/test.sh 2024-03-28T01:08:13.357 INFO:tasks.workunit.client.0.smithi033.stderr:+ ceph_test_client 2024-03-28T01:08:13.399 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] Running 25 tests from 1 test suite. 2024-03-28T01:08:13.399 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment set-up. 2024-03-28T01:08:13.399 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 25 tests from TestClient 2024-03-28T01:08:13.399 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.SingleTargetMdsCommand 2024-03-28T01:08:13.399 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.396+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:13.399 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.397+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:13.402 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.397+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 0x560b8404d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.397+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b8404fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.397+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b84052710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.397+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83ea1fe0 con 0x560b84050250 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.397+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83e53c40 con 0x560b8404acd0 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbaec0 con 0x560b8404d670 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b84052710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b84052710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49392/0 (socket says 172.21.15.33:49392) 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 -- 172.21.15.33:0/3397210473 learned_addr learned my addr 172.21.15.33:0/3397210473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 0x560b8404d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9dff200640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b8404fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 -- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 msgr2=0x560b8404fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.403 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b8404fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 -- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 msgr2=0x560b8404d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 0x560b8404d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.398+0000 7f9e00600640 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b84052f70 con 0x560b84050250 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9e00600640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b84052710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000fb0 tx=0x7f9df800daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d9ef720087dc3a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df80034e0 con 0x560b84050250 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9e0324a480 10 client.4629 Subscribing to map 'mdsmap' 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b84057340 con 0x560b84050250 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df8001780 con 0x560b84050250 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9df5600640 20 client.4629 tick 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9df5600640 20 client.4629 collect_and_send_metrics 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9df5600640 20 client.4629 collect_and_send_global_metrics 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9df5600640 5 client.4629 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:13.404 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9df5600640 20 client.4629 trim_cache size 0 max 16384 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9df5600640 20 client.4629 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8016070 con 0x560b84050250 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.399+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009050 con 0x560b84050250 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9dfde00640 1 client.4629 handle_mds_map epoch 8 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9e0324a480 20 client.4629 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9e0324a480 10 client.4629 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9e0324a480 20 client.4629 mds is 0 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9e0324a480 10 client.4629 _open_mds_session mds.0 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b8405a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b8405a740 con 0x560b84057d30 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9e0324a480 10 client.4629 waiting for session to mds.0 to open 2024-03-28T01:08:13.405 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.400+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b8405a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.406 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.401+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b8405a120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca5f98bb6ad2582 server_cookie=a55ff28166d6d31c in_seq=0 out_seq=0 2024-03-28T01:08:13.406 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.401+0000 7f9dfde00640 10 client.4629 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:13.410 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b8405a740 con 0x560b84057d30 2024-03-28T01:08:13.410 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:13.410 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 renew_caps mds.0 2024-03-28T01:08:13.410 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9de400ad50 con 0x560b84057d30 2024-03-28T01:08:13.410 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 connect_mds_targets for mds.0 2024-03-28T01:08:13.410 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 10 client.4629 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:13.410 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 20 client.4629 mds is 0 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 10 client.4629 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 20 client.4629 encode_cap_releases enter (req: 0x560b840579b0, mds: 0) 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 20 client.4629 send_request set sent_stamp to 2024-03-28T01:08:13.406862+0000 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 10 client.4629 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:13.400759+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:13.400759+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8405ae10 con 0x560b84057d30 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 20 client.4629 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9de400ad50 con 0x560b84057d30 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b84057d30 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 insert_trace from 2024-03-28T01:08:13.406862+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 features 0xffffffffffffffff 2024-03-28T01:08:13.411 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 update_snap_trace len 70 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 get_snap_realm 0x1 0x7f9de400a130, nref now is 1 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 15 client.4629 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 15 client.4629 snapc 1=[] 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 put_snap_realm 0x1 0x7f9de400a130 2 -> 1 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 hrm is_target=1 is_dentry=0 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9de400af70) - ctime 2024-03-28T01:06:22.699088+0000 mtime 2024-03-28T01:06:22.699088+0000 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 dir hash is 2 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 12 client.4629 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 12 client.4629 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=- has_dir_layout 0x7f9de400af70) caps pAsLsXsFs 2024-03-28T01:08:13.412 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 get_snap_realm 0x1 0x7f9de400a130, nref was 1 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 get_snap_realm 0x1 0x7f9de400a130, nref now is 2 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 15 client.4629 add_update_cap first one, opened snaprealm 0x7f9de400a130 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400af70) 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 add_update_cap calling signal_caps_inode 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 10 client.4629 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400af70) 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 put_snap_realm 0x1 0x7f9de400a130 2 -> 1 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 20 client.4629 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 20 client.4629 lat 0.000634 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 20 client.4629 _ll_get 0x7f9de400af70 0x1 -> 1 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.413 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9dfde00640 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.406+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9df0054400 con 0x560b84050250 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9df801e080 con 0x560b84050250 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9e0324a480 10 client.4629 fetch_fsmap learned FSMap version 8 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=8}) v3 -- 0x560b8405b1e0 con 0x560b84050250 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== fsmap(e 8) v1 ==== 2164+0+0 (secure 0 0 0) 0x7f9df8001980 con 0x560b84050250 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9e0324a480 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9e0324a480 10 client.4629 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9e0324a480 4 client.4629 mds_command: new command op to 4245 tid=0 multi_id=0 [{"prefix": "session ls", "format": "json"}] 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.407+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- command(tid 0: {"prefix": "session ls", "format": "json"}) v1 -- 0x560b83eb3480 con 0x560b84057d30 2024-03-28T01:08:13.414 INFO:tasks.workunit.client.0.smithi033.stdout:SingleTargetMdsCommand: [{"id":4629,"entity":{"name":{"type":"client","num":4629},"addr":{"type":"any","addr":"172.21.15.33:0","nonce":3397210473}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":0.0059999040000000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.9999953791034083,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.4629 172.21.15.33:0/3397210473","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3be9a5c9bc793e11e2800a8c0c696e8b46742033","ceph_version":"ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)","entity_id":"admin","hostname":"smithi033","pid":"44790","root":"/"}}] 2024-03-28T01:08:13.416 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+1171 (crc 0 0 0) 0x560b83eb3480 con 0x560b84057d30 2024-03-28T01:08:13.416 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9dfde00640 10 client.4629 handle_command_reply: tid=0 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 2 client.4629 unmount 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 2 client.4629 unmounting 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 _ll_drop_pins 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 _ll_put 0x7f9de400af70 0x1 1 -> 0 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 nothing to flush 2024-03-28T01:08:13.417 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 1 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 flush_caps_sync 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 trim_cache size 0 max 16384 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) n = 8 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400af70) 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 15 client.4629 remove_cap last one, closing snaprealm 0x7f9de400a130 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 20 client.4629 put_snap_realm 0x1 0x7f9de400a130 1 -> 0 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 10 client.4629 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=- COMPLETE has_dir_layout 0x7f9de400af70) 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 2 client.4629 _close_mds_session mds.0 seq 0 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b8405b6b0 con 0x560b84057d30 2024-03-28T01:08:13.418 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.408+0000 7f9e0324a480 2 client.4629 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83ef9580 con 0x560b84057d30 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 10 client.4629 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 5 client.4629 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 1 -- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 msgr2=0x560b8405a120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b8405a120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 10 client.4629 remove_session_caps mds.0 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 10 client.4629 kick_requests_closed for mds.0 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 10 client.4629 unmounting: trim pass, size was 0+0 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 20 client.4629 trim_cache size 0 max 16384 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9dfde00640 10 client.4629 unmounting: trim pass, size still 0+0 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 2 client.4629 unmounted. 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 client.4629 shutdown 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 msgr2=0x560b84052710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:13.419 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b84052710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000fb0 tx=0x7f9df800daf0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 shutdown_connections 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b8405a120 unknown :-1 s=CLOSED pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b84052710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b8404fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3397210473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 0x560b8404d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 >> 172.21.15.33:0/3397210473 conn(0x560b83f2d970 msgr2=0x560b83f29100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 shutdown_connections 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 1 -- 172.21.15.33:0/3397210473 wait complete. 2024-03-28T01:08:13.420 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.412+0000 7f9e0324a480 20 client.4629 trim_cache size 0 max 16384 2024-03-28T01:08:13.421 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.SingleTargetMdsCommand (16 ms) 2024-03-28T01:08:13.421 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.MultiTargetMdsCommand 2024-03-28T01:08:13.424 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.413+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:13.424 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.413+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:13.424 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b83fb4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84050250 0x560b83f29a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4008a50 con 0x560b8404acd0 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de400be10 con 0x560b84050250 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4009ad0 con 0x560b8404d670 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49402/0 (socket says 172.21.15.33:49402) 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9dffc00640 1 -- 172.21.15.33:0/3134151474 learned_addr learned my addr 172.21.15.33:0/3134151474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9dff200640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b83fb4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.414+0000 7f9e00600640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84050250 0x560b83f29a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dffc00640 1 -- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 msgr2=0x560b83fb4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b83fb4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.425 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dffc00640 1 -- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84050250 msgr2=0x560b83f29a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84050250 0x560b83f29a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dffc00640 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83ef8f40 con 0x560b8404acd0 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9df000a7e0 tx=0x7f9de4004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c394d0b82fd5d64b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0067070 con 0x560b8404acd0 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9e0324a480 10 client.4632 Subscribing to map 'mdsmap' 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df0005240 con 0x560b8404acd0 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.415+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83fbb8b0 con 0x560b8404acd0 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9df6000640 20 client.4632 tick 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9df6000640 20 client.4632 collect_and_send_metrics 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9df6000640 20 client.4632 collect_and_send_global_metrics 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9df6000640 5 client.4632 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9df6000640 20 client.4632 trim_cache size 0 max 16384 2024-03-28T01:08:13.426 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9df6000640 20 client.4632 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0005440 con 0x560b8404acd0 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0015300 con 0x560b8404acd0 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9dfca00640 1 client.4632 handle_mds_map epoch 8 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9e0324a480 20 client.4632 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9e0324a480 10 client.4632 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9e0324a480 20 client.4632 mds is 0 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9e0324a480 10 client.4632 _open_mds_session mds.0 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83fb42f0 con 0x560b84057d30 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9e0324a480 10 client.4632 waiting for session to mds.0 to open 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.416+0000 7f9dff200640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.417+0000 7f9dff200640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f98d284f620084 server_cookie=5f4e331c90c405be in_seq=0 out_seq=0 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.417+0000 7f9dfca00640 10 client.4632 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:13.427 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83fb42f0 con 0x560b84057d30 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 renew_caps mds.0 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9de400c2d0 con 0x560b84057d30 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 connect_mds_targets for mds.0 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 10 client.4632 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 20 client.4632 mds is 0 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 10 client.4632 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 20 client.4632 encode_cap_releases enter (req: 0x560b83f28370, mds: 0) 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 20 client.4632 send_request set sent_stamp to 2024-03-28T01:08:13.420893+0000 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 10 client.4632 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:13.417033+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:13.417033+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2c070 con 0x560b84057d30 2024-03-28T01:08:13.428 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 20 client.4632 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9de400c2d0 con 0x560b84057d30 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de800b410 con 0x560b84057d30 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 insert_trace from 2024-03-28T01:08:13.420893+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 features 0xffffffffffffffff 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 update_snap_trace len 70 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 get_snap_realm 0x1 0x7f9de400c4f0, nref now is 1 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 15 client.4632 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 15 client.4632 snapc 1=[] 2024-03-28T01:08:13.429 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:13.430 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 put_snap_realm 0x1 0x7f9de400c4f0 2 -> 1 2024-03-28T01:08:13.430 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 hrm is_target=1 is_dentry=0 2024-03-28T01:08:13.430 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9de400c5e0) - ctime 2024-03-28T01:06:22.699088+0000 mtime 2024-03-28T01:06:22.699088+0000 2024-03-28T01:08:13.430 INFO:tasks.workunit.client.0.smithi033.stdout:MultiTargetMds: 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 dir hash is 2 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 12 client.4632 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 12 client.4632 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=- has_dir_layout 0x7f9de400c5e0) caps pAsLsXsFs 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 get_snap_realm 0x1 0x7f9de400c4f0, nref was 1 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 get_snap_realm 0x1 0x7f9de400c4f0, nref now is 2 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 15 client.4632 add_update_cap first one, opened snaprealm 0x7f9de400c4f0 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400c5e0) 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 add_update_cap calling signal_caps_inode 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 10 client.4632 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400c5e0) 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 put_snap_realm 0x1 0x7f9de400c4f0 2 -> 1 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:13.432 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 20 client.4632 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 20 client.4632 lat 0.000443 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 20 client.4632 _ll_get 0x7f9de400c5e0 0x1 -> 1 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b83e26b70 con 0x560b8404acd0 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9df0073080 con 0x560b8404acd0 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.420+0000 7f9e0324a480 10 client.4632 fetch_fsmap learned FSMap version 8 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=8}) v3 -- 0x560b84055b30 con 0x560b8404acd0 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mon.0 v2:172.21.15.33:3300/0 6 ==== fsmap(e 8) v1 ==== 2164+0+0 (secure 0 0 0) 0x7f9df0076020 con 0x560b8404acd0 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 10 client.4632 resolve_mds: resolving `*' to all MDS daemons 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 10 client.4632 resolve_mds: appending daemon mds.b to targets 2024-03-28T01:08:13.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 10 client.4632 resolve_mds: appending daemon mds.a to targets 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:6832/214301400,v1:172.21.15.194:6833/214301400] conn(0x560b83fb02b0 0x560b83fb06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 4 client.4632 mds_command: new command op to 4232 tid=0 multi_id=1 [{"prefix": "session ls", "format": "json"}] 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.194:6832/214301400,v1:172.21.15.194:6833/214301400] -- command(tid 0: {"prefix": "session ls", "format": "json"}) v1 -- 0x560b83e77700 con 0x560b83fb02b0 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 4 client.4632 mds_command: new command op to 4245 tid=1 multi_id=1 [{"prefix": "session ls", "format": "json"}] 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- command(tid 1: {"prefix": "session ls", "format": "json"}) v1 -- 0x560b83fb0cb0 con 0x560b84057d30 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+1172 (crc 0 0 0) 0x560b83fb0cb0 con 0x560b84057d30 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.421+0000 7f9dfca00640 10 client.4632 handle_command_reply: tid=1 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.422+0000 7f9e00600640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:6832/214301400,v1:172.21.15.194:6833/214301400] conn(0x560b83fb02b0 0x560b83fb06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.422+0000 7f9e00600640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:6832/214301400,v1:172.21.15.194:6833/214301400] conn(0x560b83fb02b0 0x560b83fb06d0 crc :-1 s=READY pgs=5 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.? client_cookie=1d13082ba4af4032 server_cookie=62689be77a06349b in_seq=0 out_seq=0 2024-03-28T01:08:13.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.422+0000 7f9dfca00640 10 client.4632 ms_handle_connect on v2:172.21.15.194:6832/214301400 2024-03-28T01:08:13.435 INFO:tasks.workunit.client.0.smithi033.stdout:MultiTargetMdsCommand: [{"mds.a":[{"id":4632,"entity":{"name":{"type":"client","num":4632},"addr":{"type":"any","addr":"172.21.15.33:0","nonce":3134151474}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":0.0039999359999999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99999768954903512,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.4632 172.21.15.33:0/3134151474","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3be9a5c9bc793e11e2800a8c0c696e8b46742033","ceph_version":"ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)","entity_id":"admin","hostname":"smithi033","pid":"44790","root":"/"}}]}]�V� 2024-03-28T01:08:13.436 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mds.? v2:172.21.15.194:6832/214301400 1 ==== command_reply(tid 0: -38 ) v1 ==== 8+0+0 (crc 0 0 0) 0x560b83e77700 con 0x560b83fb02b0 2024-03-28T01:08:13.436 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9dfca00640 10 client.4632 handle_command_reply: tid=0 2024-03-28T01:08:13.436 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 2 client.4632 unmount 2024-03-28T01:08:13.436 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 2 client.4632 unmounting 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 _ll_drop_pins 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 _ll_put 0x7f9de400c5e0 0x1 1 -> 0 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 nothing to flush 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 1 2024-03-28T01:08:13.437 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 flush_caps_sync 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 trim_cache size 0 max 16384 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) n = 8 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400c5e0) 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 15 client.4632 remove_cap last one, closing snaprealm 0x7f9de400c4f0 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 20 client.4632 put_snap_realm 0x1 0x7f9de400c4f0 1 -> 0 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 10 client.4632 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=- COMPLETE has_dir_layout 0x7f9de400c5e0) 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 2 client.4632 _close_mds_session mds.0 seq 0 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83fb10d0 con 0x560b84057d30 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.423+0000 7f9e0324a480 2 client.4632 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83fb10d0 con 0x560b84057d30 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 10 client.4632 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:13.438 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 5 client.4632 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 1 -- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 msgr2=0x560b83f2b770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 10 client.4632 remove_session_caps mds.0 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 10 client.4632 kick_requests_closed for mds.0 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 10 client.4632 unmounting: trim pass, size was 0+0 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 20 client.4632 trim_cache size 0 max 16384 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 10 client.4632 unmounting: trim pass, size still 0+0 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 2 client.4632 unmounted. 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 client.4632 shutdown 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 msgr2=0x560b83fbb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9df000a7e0 tx=0x7f9de4004120 comp rx=0 tx=0).stop 2024-03-28T01:08:13.439 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 shutdown_connections 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:6832/214301400,v1:172.21.15.194:6833/214301400] conn(0x560b83fb02b0 0x560b83fb06d0 crc :-1 s=READY pgs=5 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b770 unknown :-1 s=CLOSED pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84050250 0x560b83f29a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b83fb4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e00600640 1 -- 172.21.15.33:0/3134151474 reap_dead start 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9dfca00640 0 client.4632 ms_handle_reset on v2:172.21.15.194:6832/214301400 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3134151474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb1f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 >> 172.21.15.33:0/3134151474 conn(0x560b83f2d970 msgr2=0x560b83f2ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 shutdown_connections 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 1 -- 172.21.15.33:0/3134151474 wait complete. 2024-03-28T01:08:13.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.427+0000 7f9e0324a480 20 client.4632 trim_cache size 0 max 16384 2024-03-28T01:08:13.441 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.MultiTargetMdsCommand (15 ms) 2024-03-28T01:08:13.441 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritev 2024-03-28T01:08:13.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.428+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.428+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404d670 0x560b83f2e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84050250 0x560b83fba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4005990 con 0x560b8404acd0 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de400d580 con 0x560b8404d670 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4008360 con 0x560b84050250 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49404/0 (socket says 172.21.15.33:49404) 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9dffc00640 1 -- 172.21.15.33:0/1640151560 learned_addr learned my addr 172.21.15.33:0/1640151560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9e00600640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84050250 0x560b83fba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.429+0000 7f9dff200640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404d670 0x560b83f2e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dffc00640 1 -- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84050250 msgr2=0x560b83fba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84050250 0x560b83fba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dffc00640 1 -- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404d670 msgr2=0x560b83f2e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404d670 0x560b83f2e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dffc00640 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b84055740 con 0x560b8404acd0 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dff200640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404d670 0x560b83f2e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9df00501e0 tx=0x7f9df00634a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d4b443bdbcf9ce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0067070 con 0x560b8404acd0 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df0054cb0 con 0x560b8404acd0 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9e0324a480 10 client.4635 Subscribing to map 'mdsmap' 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b84057690 con 0x560b8404acd0 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9df5600640 20 client.4635 tick 2024-03-28T01:08:13.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9df5600640 20 client.4635 collect_and_send_metrics 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9df5600640 20 client.4635 collect_and_send_global_metrics 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9df5600640 5 client.4635 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9df5600640 20 client.4635 trim_cache size 0 max 16384 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9df5600640 20 client.4635 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.430+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b83e26b70 con 0x560b8404acd0 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0070070 con 0x560b8404acd0 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9dfde00640 1 client.4635 handle_mds_map epoch 8 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9e0324a480 20 client.4635 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9e0324a480 10 client.4635 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9e0324a480 20 client.4635 mds is 0 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9e0324a480 10 client.4635 _open_mds_session mds.0 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83fb45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:13.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83fbeab0 con 0x560b84057d30 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9e0324a480 10 client.4635 waiting for session to mds.0 to open 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9dff200640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83fb45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9dfde00640 10 client.4635 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.431+0000 7f9dff200640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83fb45f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76a9ab591ff68e3 server_cookie=c6ce92182c7bdf5a in_seq=0 out_seq=0 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83fbeab0 con 0x560b84057d30 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 renew_caps mds.0 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9de400c560 con 0x560b84057d30 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 connect_mds_targets for mds.0 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:13.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 mds is 0 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 encode_cap_releases enter (req: 0x560b83fb4270, mds: 0) 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 send_request set sent_stamp to 2024-03-28T01:08:13.435944+0000 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:13.431731+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:13.431731+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fbf100 con 0x560b84057d30 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9de400c560 con 0x560b84057d30 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de8018040 con 0x560b84057d30 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 insert_trace from 2024-03-28T01:08:13.435944+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 features 0xffffffffffffffff 2024-03-28T01:08:13.450 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 update_snap_trace len 70 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 get_snap_realm 0x1 0x7f9de400c780, nref now is 1 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 15 client.4635 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 15 client.4635 snapc 1=[] 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 put_snap_realm 0x1 0x7f9de400c780 2 -> 1 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 hrm is_target=1 is_dentry=0 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9de400d700) - ctime 2024-03-28T01:06:22.699088+0000 mtime 2024-03-28T01:06:22.699088+0000 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 dir hash is 2 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 12 client.4635 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:13.451 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 12 client.4635 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=- has_dir_layout 0x7f9de400d700) caps pAsLsXsFs 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 get_snap_realm 0x1 0x7f9de400c780, nref was 1 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 get_snap_realm 0x1 0x7f9de400c780, nref now is 2 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 15 client.4635 add_update_cap first one, opened snaprealm 0x7f9de400c780 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400d700) 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 add_update_cap calling signal_caps_inode 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400d700) 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 put_snap_realm 0x1 0x7f9de400c780 2 -> 1 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 lat 0.000473 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 _ll_get 0x7f9de400d700 0x1 -> 1 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 2 client.4635 unmount 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 2 client.4635 unmounting 2024-03-28T01:08:13.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83fbf450 con 0x560b84057d30 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 _ll_drop_pins 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 _ll_put 0x7f9de400d700 0x1 1 -> 0 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 nothing to flush 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 flush_caps_sync 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 20 client.4635 trim_cache size 0 max 16384 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 10 client.4635 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 6 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 2 client.4635 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 1 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 unmounting: trim pass, size was 0+1 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 20 client.4635 trim_cache size 0 max 16384 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9dfde00640 10 client.4635 unmounting: trim pass, size still 0+1 2024-03-28T01:08:13.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:13.435+0000 7f9e0324a480 2 client.4635 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:14.433 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.430+0000 7f9df5600640 20 client.4635 tick 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.430+0000 7f9df5600640 10 client.4635 renew_caps() 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.430+0000 7f9df5600640 15 client.4635 renew_caps requesting from mds.0 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.430+0000 7f9df5600640 10 client.4635 renew_caps mds.0 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.430+0000 7f9df5600640 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 2) v7 -- 0x7f9ddc001de0 con 0x560b84057d30 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 20 client.4635 collect_and_send_metrics 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 20 client.4635 collect_and_send_global_metrics 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_metrics [client_metric_type: READ_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: WRITE_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: METADATA_LATENCY latency: 0.000473, avg_latency: 0.000473, sq_sum: 0, count=1][client_metric_type: CAP_INFO cap_hits: 0 cap_misses: 0 num_caps: 0][client_metric_type: DENTRY_LEASE dlease_hits: 0 dlease_misses: 0 num_dentries: 0][client_metric_type: OPENED_FILES opened_files: 0 total_inodes: 1][client_metric_type: PINNED_ICAPS pinned_icaps: 1 total_inodes: 1][client_metric_type: OPENED_INODES opened_inodes: 0 total_inodes: 1][client_metric_type: READ_IO_SIZES total_ops: 0 total_size: 0][client_metric_type: WRITE_IO_SIZES total_ops: 0 total_size: 0] v1 -- 0x7f9ddc002670 con 0x560b84057d30 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 10 client.4635 _put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) n = 2 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 10 client.4635 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400d700) 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 15 client.4635 remove_cap last one, closing snaprealm 0x7f9de400c780 2024-03-28T01:08:14.434 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 20 client.4635 put_snap_realm 0x1 0x7f9de400c780 1 -> 0 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 10 client.4635 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=- COMPLETE has_dir_layout 0x7f9de400d700) 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 20 client.4635 trim_cache size 0 max 16384 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9df5600640 20 client.4635 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9e0324a480 2 client.4635 _close_mds_session mds.0 seq 0 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b840612b0 con 0x560b84057d30 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9e0324a480 2 client.4635 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9ddc001de0 con 0x560b84057d30 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9dfde00640 10 client.4635 handle_client_session client_session(renewcaps seq 2) v1 from mds.0 2024-03-28T01:08:14.435 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9dfde00640 10 client.4635 unmounting: trim pass, size was 0+0 2024-03-28T01:08:14.436 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9dfde00640 20 client.4635 trim_cache size 0 max 16384 2024-03-28T01:08:14.436 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.431+0000 7f9dfde00640 10 client.4635 unmounting: trim pass, size still 0+0 2024-03-28T01:08:14.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.436+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b840612b0 con 0x560b84057d30 2024-03-28T01:08:14.440 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.436+0000 7f9dfde00640 10 client.4635 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.436+0000 7f9dfde00640 5 client.4635 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.436+0000 7f9dfde00640 1 -- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 msgr2=0x560b83fb45f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.436+0000 7f9dfde00640 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83fb45f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9dfde00640 10 client.4635 remove_session_caps mds.0 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9dfde00640 10 client.4635 kick_requests_closed for mds.0 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9dfde00640 10 client.4635 unmounting: trim pass, size was 0+0 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9dfde00640 20 client.4635 trim_cache size 0 max 16384 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9dfde00640 10 client.4635 unmounting: trim pass, size still 0+0 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9e0324a480 2 client.4635 unmounted. 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9e0324a480 1 client.4635 shutdown 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 msgr2=0x560b83fbb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9df00501e0 tx=0x7f9df00634a0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 shutdown_connections 2024-03-28T01:08:14.441 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.437+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83fb45f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.442 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.438+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84050250 0x560b83fba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.442 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.438+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404d670 0x560b83f2e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.442 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.438+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1640151560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8404acd0 0x560b83fbb070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.442 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.438+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 >> 172.21.15.33:0/1640151560 conn(0x560b83f2d970 msgr2=0x560b83f2ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:14.442 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.438+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 shutdown_connections 2024-03-28T01:08:14.442 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.438+0000 7f9e0324a480 1 -- 172.21.15.33:0/1640151560 wait complete. 2024-03-28T01:08:14.442 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.438+0000 7f9e0324a480 20 client.4635 trim_cache size 0 max 16384 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.439+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.440+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.440+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 0x560b83fba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.440+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b84055040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.440+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b83f2c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.440+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4009360 con 0x560b84050250 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.440+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8405b530 con 0x560b8404acd0 2024-03-28T01:08:14.443 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.440+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8405b9a0 con 0x560b8404d670 2024-03-28T01:08:14.444 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.441+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b83f2c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.444 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.441+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b83f2c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49420/0 (socket says 172.21.15.33:49420) 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.441+0000 7f9e00600640 1 -- 172.21.15.33:0/2977342527 learned_addr learned my addr 172.21.15.33:0/2977342527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.441+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 0x560b83fba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.441+0000 7f9dff200640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b84055040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.442+0000 7f9e00600640 1 -- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 msgr2=0x560b84055040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.442+0000 7f9e00600640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8404d670 0x560b84055040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.442+0000 7f9e00600640 1 -- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 msgr2=0x560b83fba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.442+0000 7f9e00600640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404acd0 0x560b83fba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.445 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.442+0000 7f9e00600640 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83fb1d20 con 0x560b84050250 2024-03-28T01:08:14.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.442+0000 7f9e00600640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b83f2c980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9df800db30 tx=0x7f9df8001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413d002192b477b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:14.446 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.442+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800d480 con 0x560b84050250 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df8030b20 con 0x560b84050250 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 10 client.4638 Subscribing to map 'mdsmap' 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83f29c00 con 0x560b84050250 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9df6000640 20 client.4638 tick 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9df6000640 20 client.4638 collect_and_send_metrics 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9df6000640 20 client.4638 collect_and_send_global_metrics 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9df6000640 5 client.4638 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9df6000640 20 client.4638 trim_cache size 0 max 16384 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9df6000640 20 client.4638 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0054400 con 0x560b84050250 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b84050250 2024-03-28T01:08:14.447 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9dfca00640 1 client.4638 handle_mds_map epoch 8 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 20 client.4638 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 10 client.4638 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 20 client.4638 mds is 0 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 10 client.4638 _open_mds_session mds.0 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.443+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f2b9b0 con 0x560b84057d30 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.444+0000 7f9e0324a480 10 client.4638 waiting for session to mds.0 to open 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.444+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.448 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.445+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8270f1e58070ae2f server_cookie=38388bf5aff81414 in_seq=0 out_seq=0 2024-03-28T01:08:14.449 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.445+0000 7f9dfca00640 10 client.4638 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:14.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f2b9b0 con 0x560b84057d30 2024-03-28T01:08:14.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9dfca00640 10 client.4638 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:14.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9dfca00640 10 client.4638 renew_caps mds.0 2024-03-28T01:08:14.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9de400d700 con 0x560b84057d30 2024-03-28T01:08:14.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9dfca00640 10 client.4638 connect_mds_targets for mds.0 2024-03-28T01:08:14.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 10 client.4638 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:14.452 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 20 client.4638 mds is 0 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 10 client.4638 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 20 client.4638 encode_cap_releases enter (req: 0x560b83f2b100, mds: 0) 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 20 client.4638 send_request set sent_stamp to 2024-03-28T01:08:14.449560+0000 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 10 client.4638 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:14.444616+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:14.444616+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84061590 con 0x560b84057d30 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.448+0000 7f9e0324a480 20 client.4638 awaiting reply|forward|kick on 0x7ffe682696e0 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9de400d700 con 0x560b84057d30 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b84057d30 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 insert_trace from 2024-03-28T01:08:14.449560+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 features 0xffffffffffffffff 2024-03-28T01:08:14.453 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 update_snap_trace len 70 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref now is 1 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 15 client.4638 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 15 client.4638 snapc 1=[] 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 put_snap_realm 0x1 0x7f9de400d920 2 -> 1 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 hrm is_target=1 is_dentry=0 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9de400da10) - ctime 2024-03-28T01:06:22.699088+0000 mtime 2024-03-28T01:06:22.699088+0000 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 dir hash is 2 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 12 client.4638 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 12 client.4638 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=- has_dir_layout 0x7f9de400da10) caps pAsLsXsFs 2024-03-28T01:08:14.454 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref was 1 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref now is 2 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 15 client.4638 add_update_cap first one, opened snaprealm 0x7f9de400d920 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 add_update_cap calling signal_caps_inode 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 10 client.4638 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 put_snap_realm 0x1 0x7f9de400d920 2 -> 1 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 handle_client_reply signalling caller 0x7ffe682696e0 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 lat 0.000596 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 _ll_get 0x7f9de400da10 0x1 -> 1 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9dfca00640 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.455 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 8 client.4638 _ll_create 0x1.head test_llreadvllwritevfile44790 0666 578, uid 0, gid 0 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 10 client.4638 _lookup concluded ENOENT locally for 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) dn 'test_llreadvllwritevfile44790' 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 may_create 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 10 client.4638 _getattr mask As issued=1 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 3 client.4638 may_create 0x7f9de400da10 = 0 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 8 client.4638 _create(0x1 test_llreadvllwritevfile44790, 0666) 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 get_or_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) name test_llreadvllwritevfile44790 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 15 open_dir 0x560b84061400 on 0x7f9de400da10 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 15 client.4638 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) 'test_llreadvllwritevfile44790' dn 0x1.head["test_llreadvllwritevfile44790"] (new dn) 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 choose_target_mds inode dir hash is 2 on test_llreadvllwritevfile44790 => 3273908422 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 choose_target_mds 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) is_hash=1 hash=3273908422 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 10 client.4638 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 mds is 0 2024-03-28T01:08:14.456 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 10 client.4638 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:14.457 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 encode_cap_releases enter (req: 0x560b840618c0, mds: 0) 2024-03-28T01:08:14.457 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 encode_dentry_release enter(dn:0x7f9de400a130) 2024-03-28T01:08:14.457 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9de400da10), req:0x560b840618c0 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:14.457 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 send_request set sent_stamp to 2024-03-28T01:08:14.450319+0000 2024-03-28T01:08:14.457 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 10 client.4638 send_request client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevfile44790 2024-03-28T01:08:14.450279+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:14.457 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevfile44790 2024-03-28T01:08:14.450279+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84061d40 con 0x560b84057d30 2024-03-28T01:08:14.457 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.449+0000 7f9e0324a480 20 client.4638 awaiting reply|forward|kick on 0x7ffe68269520 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 850+0+0 (crc 0 0 0) 0x7f9df0002030 con 0x560b84057d30 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 insert_trace from 2024-03-28T01:08:14.450319+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 features 0xffffffffffffffff 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 update_snap_trace len 70 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref was 1 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref now is 2 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 hrm is_target=1 is_dentry=1 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 update_inode_file_time 0x10000000000.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) - ctime 2024-03-28T01:08:14.450279+0000 mtime 2024-03-28T01:08:14.450279+0000 2024-03-28T01:08:14.459 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 12 client.4638 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 12 client.4638 add_update_inode adding 0x10000000000.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=- objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref was 2 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref now is 3 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 15 client.4638 add_update_cap first one, opened snaprealm 0x7f9de400d920 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000000.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 add_update_cap calling signal_caps_inode 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 12 client.4638 add_update_inode had 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9de400da10) caps pAsLsXsFs 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 update_inode_file_time 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:06:22.699088+0000 ctime=2024-03-28T01:06:22.699088+0000 change_attr=0 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9de400da10) pAsLsXs ctime 2024-03-28T01:08:14.450279+0000 mtime 2024-03-28T01:08:14.450279+0000 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 dir hash is 2 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 12 client.4638 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 1 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 clearing (I_COMPLETE|I_DIR_ORDERED) on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 10 client.4638 add_update_cap calling signal_caps_inode 2024-03-28T01:08:14.460 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 12 client.4638 insert_dentry_inode 'test_llreadvllwritevfile44790' vino 0x10000000000.head in dir 0x1.head dn 0x7f9de400a130 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 15 client.4638 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) 'test_llreadvllwritevfile44790' to inode 0x7f9de400dff0 dn 0x1.head["test_llreadvllwritevfile44790"] (old dn) 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 link inode 0x7f9de400dff0 parents now 0x1.head["test_llreadvllwritevfile44790"] 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 15 client.4638 update_dentry_lease 0x1.head["test_llreadvllwritevfile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:14.450319+0000 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 put_snap_realm 0x1 0x7f9de400d920 3 -> 2 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 handle_client_reply signalling caller 0x7ffe68269520 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9dfca00640 20 client.4638 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 20 client.4638 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 10 client.4638 delegated_inos: [] 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 10 client.4638 make_request created ino 0x10000000000 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 20 client.4638 make_request target is 0x10000000000.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.461 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 20 client.4638 lat 0.005262 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000000.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000000.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 10 client.4638 _create_fh 0x10000000000 mode 3 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 20 client.4638 trim_cache size 1 max 16384 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 8 client.4638 create(#0x1/test_llreadvllwritevfile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 20 client.4638 _ll_create created = 1 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 8 client.4638 _ll_create 0x1.head test_llreadvllwritevfile44790 0666 578 = 0 (0x560b83e34140 10000000000) 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 20 client.4638 _ll_get 0x7f9de400dff0 0x10000000000 -> 1 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.454+0000 7f9e0324a480 10 client.4638 fill_statx on 0x10000000000 snap/devhead mode 0100666 mtime 2024-03-28T01:08:14.450279+0000 ctime 2024-03-28T01:08:14.450279+0000 change_attr 0 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9e0324a480 10 client.4638 get_quota_root realm 0x1 2024-03-28T01:08:14.462 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9e0324a480 10 client.4638 get_quota_root 0x10000000000.head -> 0x1.head 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9e0324a480 10 client.4638 cur file size is 0 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x560b83fab840 0x560b83fadcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] -- osd_op(unknown.0.0:1 3.30 3:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x7f9de400cd80 con 0x560b83fab840 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] -- osd_op(unknown.0.0:2 3.30 3:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x7f9de400b5f0 con 0x560b83fab840 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9dfca00640 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.455+0000 7f9dff200640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x560b83fab840 0x560b83fadcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.456+0000 7f9dfca00640 10 client.4638 ms_handle_connect on v2:172.21.15.194:6816/3192443429 2024-03-28T01:08:14.463 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.456+0000 7f9dff200640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x560b83fab840 0x560b83fadcd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=5e7370875ff54526 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.457+0000 7f9dff200640 1 -- 172.21.15.33:0/2977342527 <== osd.5 v2:172.21.15.194:6816/3192443429 1 ==== osd_op_reply(1 10000000000.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de801e410 con 0x560b83fab840 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9dff200640 1 -- 172.21.15.33:0/2977342527 <== osd.5 v2:172.21.15.194:6816/3192443429 2 ==== osd_op_reply(2 10000000000.00000000 [create] v22'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de801e410 con 0x560b83fab840 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 wanted_max_size 0 -> 14 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_caps 0x10000000000.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 20 client.4638 clear_suid_sgid 0x10000000000.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={4=0,4096=1} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 5 client.4638 get_cap_ref got first FILE_BUFFER ref on 0x10000000000.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={4=0,4096=1,8192=0} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 5 client.4638 get_cap_ref got first FILE_CACHE ref on 0x10000000000.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={4=0,1024=0,4096=1,8192=0} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 3 client.4638 pwritev(0x560b83e34140, "...", 14, 0) = 0 2024-03-28T01:08:14.466 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 mark_caps_dirty 0x10000000000.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) - -> Fw 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 cap_delay_requeue on 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 get_quota_root realm 0x1 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 get_quota_root 0x10000000000.head -> 0x1.head 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 7 client.4638 wrote to 14, extending file size 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 mark_caps_dirty 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) Fw -> Fw 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 cap_delay_requeue on 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 19 client.4638 C_Write_Finisher::try_complete this 0x560b83e434d0 onuninlinefinished 1 iofinished 1 iofinished_r 14 fsync_finished 1 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 19 client.4638 complete with iofinished_r 14 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_caps 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=0,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) have pAsxLsXsxFsxcrwb need Fr want Fc revoking - 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 _read_async 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=1,4096=0,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 0~14 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 min_bytes=131072 max_bytes=16777216 max_periods=4 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 3 client.4638 preadv(0x560b83e34140, 0) = 0 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_quota_root realm 0x1 2024-03-28T01:08:14.467 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_quota_root 0x10000000000.head -> 0x1.head 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 cur file size is 14 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 wanted_max_size 14 -> 126 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_caps 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 20 client.4638 clear_suid_sgid 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 3 client.4638 pwritev(0x560b83e34140, "...", 26, 100) = 0 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 mark_caps_dirty 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) Fw -> Fw 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 cap_delay_requeue on 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461108+0000 ctime=2024-03-28T01:08:14.461108+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 get_quota_root realm 0x1 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 get_quota_root 0x10000000000.head -> 0x1.head 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 7 client.4638 wrote to 126, extending file size 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 mark_caps_dirty 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) Fw -> Fw 2024-03-28T01:08:14.468 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 cap_delay_requeue on 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 19 client.4638 C_Write_Finisher::try_complete this 0x560b83e40c70 onuninlinefinished 1 iofinished 1 iofinished_r 26 fsync_finished 1 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 19 client.4638 complete with iofinished_r 26 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_caps 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) have pAsxLsXsxFsxcrwb need Fr want Fc revoking - 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 _read_async 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=1,4096=0,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 100~26 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 min_bytes=131072 max_bytes=16777216 max_periods=4 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 3 client.4638 preadv(0x560b83e34140, 100) = 0 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_quota_root realm 0x1 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_quota_root 0x10000000000.head -> 0x1.head 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 cur file size is 126 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 wanted_max_size 126 -> 1031 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 get_caps 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 20 client.4638 clear_suid_sgid 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=126/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 40] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 10 client.4638 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e0324a480 3 client.4638 pwritev(0x560b83e34140, "...", 31, 1000) = 0 2024-03-28T01:08:14.469 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 mark_caps_dirty 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) Fw -> Fw 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 cap_delay_requeue on 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461279+0000 ctime=2024-03-28T01:08:14.461279+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 get_quota_root realm 0x1 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 get_quota_root 0x10000000000.head -> 0x1.head 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 7 client.4638 wrote to 1031, extending file size 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 mark_caps_dirty 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) Fw -> Fw 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 cap_delay_requeue on 0x10000000000.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 19 client.4638 C_Write_Finisher::try_complete this 0x560b83e72170 onuninlinefinished 1 iofinished 1 iofinished_r 31 fsync_finished 0 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 19 client.4638 kickoff fsync onfinish 0x560b84061390 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 15 client.4638 Client::C_nonblocking_fsync_state::advance progress 0 flush_wait 0 flush_completed 0 result 0 waitfor_safe 0 onfinish 0x560b840621e0 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 15 client.4638 Client::C_nonblocking_fsync_state::advance - case 0 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 _flush 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] -- osd_op(unknown.0.0:3 3.30 3:0cf6ad0f:::10000000000.00000000:head [write 0~14 in=14b,write 100~26 in=26b,write 1000~31 in=31b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x7f9dcc002840 con 0x560b83fab840 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 15 client.4638 using return-valued form of _fsync 2024-03-28T01:08:14.470 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 check_caps on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) wanted pAsxXsxFsxcrwb used Fcwb issued pAsxLsXsxFsxcrwb revoking - flags=3 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 mark_caps_flushing Fw 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 mark_caps_clean 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 10 client.4638 send_cap 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 71] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) mds.0 seq 1 used Fcwb want pAsxXsxFsxcrwb flush Fw retain pAsxLsxXsxFsxcrwbl held pAsxLsXsxFsxcrwb revoking - dropping - 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 15 client.4638 auth cap, requesting max_size 1031 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.460+0000 7f9e02600640 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000000 1 seq 1 tid 2 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 1031/1031 ts 1/18446744073709551615 mtime 2024-03-28T01:08:14.461471+0000 ctime 2024-03-28T01:08:14.461471+0000 change_attr 3) v12 -- 0x7f9dcc002f90 con 0x560b84057d30 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.461+0000 7f9e02600640 15 client.4638 Client::C_nonblocking_fsync_state::advance - case 1 2024-03-28T01:08:14.471 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.461+0000 7f9e02600640 15 client.4638 waiting on data to flush 2024-03-28T01:08:14.475 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9dff200640 1 -- 172.21.15.33:0/2977342527 <== osd.5 v2:172.21.15.194:6816/3192443429 3 ==== osd_op_reply(3 10000000000.00000000 [write 0~14,write 100~26,write 1000~31] v22'2 uv2 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f9de801e410 con 0x560b83fab840 2024-03-28T01:08:14.475 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 10 client.4638 _flushed 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=1,8192=1} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 5 client.4638 put_cap_ref dropped last FILE_BUFFER ref on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=1,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 5 client.4638 put_cap_ref dropped last FILE_CACHE ref on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=1,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=1,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 2 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 complete_flush r 0 progress 2 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 Client::C_nonblocking_fsync_state::advance progress 2 flush_wait 1 flush_completed 1 result 0 waitfor_safe 0 onfinish 0x560b840621e0 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 Client::C_nonblocking_fsync_state::advance - case 2 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 got 0 from flush writeback 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 Client::C_nonblocking_fsync_state::advance - case 3 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 in->flushing_cap_tids.empty() 0 in->flushing_caps 4096 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 Client::C_nonblocking_fsync_state::advance - flushing_caps 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 15 client.4638 Client::C_nonblocking_fsync_state::advance it->first 2 flush_tid 2 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 10 client.4638 advance on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=1,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) flushing Fw flush_tid 2 last 2 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.464+0000 7f9df7400640 10 client.4638 Adding onfinish 0x560b840621e0 for C_nonblocking_fsync_state 0x7f9dcc001d60 2024-03-28T01:08:14.476 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_caps(flush_ack ino 0x10000000000 1 seq 1 tid 2 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dcc002f90 con 0x560b84057d30 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 5 client.4638 set_cap_epoch_barrier epoch = 20 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 10 client.4638 mds.0 seq now 1 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 5 client.4638 handle_cap_flush_ack mds.0 cleaned Fw on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=1,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) with Fw 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 5 client.4638 flushing_caps Fw -> - 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 10 client.4638 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=1,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) !flushing 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 10 client.4638 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 15 client.4638 C_nonblocking_fsync_state_advancer::finish r 0 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 15 client.4638 Client::C_nonblocking_fsync_state::advance progress 4 flush_wait 1 flush_completed 1 result 0 waitfor_safe 0 onfinish 0x560b840621e0 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 15 client.4638 Client::C_nonblocking_fsync_state::advance - case 4 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 15 client.4638 in->flushing_cap_tids.empty() 1 in->flushing_caps 0 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 10 client.4638 ino 0x10000000000 has no uncommitted writes 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 3 client.4638 finish_fsync r = 0 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 19 client.4638 C_Write_Finisher::try_complete this 0x560b83e72170 onuninlinefinished 1 iofinished 1 iofinished_r 31 fsync_finished 1 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 19 client.4638 complete with iofinished_r 31 2024-03-28T01:08:14.477 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 get_caps 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) have pAsxLsXsxFsxcrwb need Fr want Fc revoking - 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 _read_async 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=1,4096=0,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 1000~31 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 min_bytes=131072 max_bytes=16777216 max_periods=4 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 3 client.4638 preadv(0x560b83e34140, 1000) = 0 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 3 client.4638 ll_release (fh)0x560b83e34140 0x10000000000 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 8 client.4638 _release_fh 0x560b83e34140 mode 3 on 0x10000000000.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=1} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 _flush 0x10000000000.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 nothing to flush 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 check_caps on 0x10000000000.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) wanted - used Fc issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:14.478 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 delaying cap release 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 cap_delay_requeue on 0x10000000000.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 _release_fh 0x560b83e34140 on inode 0x10000000000.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) no async_err state 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 3 client.4638 ll_unlink 0x1.head test_llreadvllwritevfile44790 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 20 client.4638 may_delete 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10); ; name test_llreadvllwritevfile44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 10 client.4638 _getattr mask As issued=1 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 3 client.4638 may_delete 0x7f9de400da10 = 0 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 8 client.4638 _unlink(0x1 test_llreadvllwritevfile44790 uid 0 gid 0) 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 20 client.4638 get_or_create 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) name test_llreadvllwritevfile44790 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.465+0000 7f9e0324a480 20 client.4638 _lookup have 0x1.head["test_llreadvllwritevfile44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 10 client.4638 _lookup 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) test_llreadvllwritevfile44790 = 0x10000000000.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000000.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.479 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000000.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 choose_target_mds inode dir hash is 2 on test_llreadvllwritevfile44790 => 3273908422 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 choose_target_mds 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10) is_hash=1 hash=3273908422 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 10 client.4638 choose_target_mds from dirfragtree hash 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 mds is 0 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 10 client.4638 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 encode_cap_releases enter (req: 0x560b83fafa40, mds: 0) 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 encode_inode_release enter(in:0x10000000000.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0), req:0x560b83fafa40 mds:0, drop:LsxXx, unless:-, force:0) 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 encode_dentry_release enter(dn:0x7f9de400a130) 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9de400da10), req:0x560b83fafa40 mds:0, drop:Fs, unless:Fx, force:1) 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 send_request set sent_stamp to 2024-03-28T01:08:14.466896+0000 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 10 client.4638 send_request client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevfile44790 2024-03-28T01:08:14.466782+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:14.480 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevfile44790 2024-03-28T01:08:14.466782+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8406bac0 con 0x560b84057d30 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9e0324a480 20 client.4638 awaiting reply|forward|kick on 0x7ffe68269700 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(revoke ino 0x10000000000 1 seq 2 caps=pAsXsFsxcrwb dirty=- wanted=- follows 0 size 1031/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:14.461471+0000 ctime 2024-03-28T01:08:14.461471+0000 change_attr 3) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081030 con 0x560b84057d30 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9dfca00640 10 client.4638 mds.0 seq now 2 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9dfca00640 5 client.4638 handle_cap_grant on in 0x10000000000 mds.0 seq 2 caps now pAsXsFsxcrwb was pAsxXsFsxcrwb 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9dfca00640 10 client.4638 update_inode_file_time 0x10000000000.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) pAsxXsFsxcrwb ctime 2024-03-28T01:08:14.461471+0000 mtime 2024-03-28T01:08:14.461471+0000 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9dfca00640 10 client.4638 revocation of Ax 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.466+0000 7f9dfca00640 10 client.4638 check_caps on 0x10000000000.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) wanted - used Fc issued pAsXsFsxcrwb revoking Ax flags=1 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.467+0000 7f9dfca00640 10 client.4638 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsFsxcrwb revoking Ax 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.467+0000 7f9dfca00640 10 client.4638 completed revocation of Ax 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.467+0000 7f9dfca00640 10 client.4638 send_cap 0x10000000000.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevfile44790"] 0x7f9de400dff0) mds.0 seq 2 used Fc want - flush - retain pAsLsXsFsc held pAsxXsFsxcrwb revoking Ax dropping Fxrwb 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.467+0000 7f9dfca00640 15 client.4638 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.467+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000000 1 seq 2 caps=pAsXsFsc dirty=- wanted=- follows 0 size 1031/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:14.461471+0000 ctime 2024-03-28T01:08:14.461471+0000 change_attr 3) v12 -- 0x7f9dcc002f90 con 0x560b84057d30 2024-03-28T01:08:14.481 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.467+0000 7f9dfca00640 10 client.4638 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:3 = 0 (0) Success unsafe) v1 ==== 498+0+0 (crc 0 0 0) 0x7f9df0005ad0 con 0x560b84057d30 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 handle_client_reply got a reply. Safe:0 tid 3 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 10 client.4638 insert_trace from 2024-03-28T01:08:14.466896+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 10 client.4638 features 0xffffffffffffffff 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 10 client.4638 update_snap_trace len 70 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref was 2 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 get_snap_realm 0x1 0x7f9de400d920, nref now is 3 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 10 client.4638 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 10 client.4638 hrm is_target=0 is_dentry=1 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 12 client.4638 add_update_inode had 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) caps pAsLsXs 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 10 client.4638 update_inode_file_time 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.450279+0000 ctime=2024-03-28T01:08:14.450279+0000 change_attr=1 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) pAsLsXs ctime 2024-03-28T01:08:14.466782+0000 mtime 2024-03-28T01:08:14.466782+0000 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 dir hash is 2 2024-03-28T01:08:14.482 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 12 client.4638 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 2 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 10 client.4638 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 15 client.4638 unlink dir 0x7f9de400da10 'test_llreadvllwritevfile44790' dn 0x7f9de400a130 inode 0x7f9de400dff0 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsXsFsc(0=pAsXsFsc) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 unlink inode 0x7f9de400dff0 parents now 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsXsFsc(0=pAsXsFsc) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 put_snap_realm 0x1 0x7f9de400d920 3 -> 2 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 handle_client_reply signalling caller 0x7ffe68269700 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9dfca00640 20 client.4638 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 20 client.4638 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 20 client.4638 lat 0.003332 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 20 client.4638 trim_cache size 1 max 16384 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 8 client.4638 unlink(#0x1/test_llreadvllwritevfile44790) = 0 2024-03-28T01:08:14.483 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=1 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsXsFsc(0=pAsXsFsc) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.484 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 2 client.4638 unmount 2024-03-28T01:08:14.484 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 2 client.4638 unmounting 2024-03-28T01:08:14.484 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83faafe0 con 0x560b84057d30 2024-03-28T01:08:14.484 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.469+0000 7f9e0324a480 10 client.4638 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:14.488 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.475+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_caps(grant ino 0x10000000000 1 seq 3 caps=pAsLsXsFsxcrwb dirty=- wanted=Ls follows 0 size 1031/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:14.461471+0000 ctime 2024-03-28T01:08:14.466782+0000 change_attr 4) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dcc002f90 con 0x560b84057d30 2024-03-28T01:08:14.488 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.475+0000 7f9dfca00640 10 client.4638 mds.0 seq now 3 2024-03-28T01:08:14.488 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.475+0000 7f9dfca00640 5 client.4638 handle_cap_grant on in 0x10000000000 mds.0 seq 3 caps now pAsLsXsFsxcrwb was pAsXsFsc 2024-03-28T01:08:14.488 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.475+0000 7f9dfca00640 10 client.4638 update_inode_file_time 0x10000000000.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/4194304 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.461471+0000 change_attr=3 caps=pAsXsFsc(0=pAsXsFsc) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) pAsXsFsc ctime 2024-03-28T01:08:14.466782+0000 mtime 2024-03-28T01:08:14.461471+0000 2024-03-28T01:08:14.488 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 max_size 4194304 -> 0 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 grant, new caps are LsFxrwb 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 5 client.4638 _try_to_trim_inode in 0x10000000000.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, size was 1+2 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 20 client.4638 trim_cache size 1 max 16384 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, size still 1+2 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df001ebf0 con 0x560b84057d30 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 20 client.4638 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 insert_trace from 2024-03-28T01:08:14.466896+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, size was 1+2 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 20 client.4638 trim_cache size 1 max 16384 2024-03-28T01:08:14.489 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 15 client.4638 trim_dentry unlinking dn test_llreadvllwritevfile44790 in dir 0x1 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 15 client.4638 unlink dir 0x7f9de400da10 'test_llreadvllwritevfile44790' dn 0x7f9de400a130 inode 0 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 15 client.4638 unlink removing 'test_llreadvllwritevfile44790' dn 0x7f9de400a130 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 15 client.4638 close_dir dir 0x560b84061400 on 0x7f9de400da10 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _ll_drop_pins 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 _ll_put 0x7f9de400dff0 0x10000000000 1 -> 0 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 _ll_put 0x7f9de400da10 0x1 2 -> 0 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.490 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.491 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.491 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 _release 0x10000000000.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) 2024-03-28T01:08:14.491 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _invalidate_inode_cache 0x10000000000.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9de400dff0) 2024-03-28T01:08:14.491 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _flush 0x10000000000.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 nothing to flush 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 _release 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _flush 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 nothing to flush 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x10000000000.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) n = 1 2024-03-28T01:08:14.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 1 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 flush_caps_sync 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 wait_sync_caps want 2 (last is 2, 0 total flushing) 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 trim_cache size 0 max 16384 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) n = 11 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 15 client.4638 remove_cap last one, closing snaprealm 0x7f9de400d920 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_snap_realm 0x1 0x7f9de400d920 2 -> 1 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=- has_dir_layout 0x7f9de400da10) 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _put_inode on 0x10000000000.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) n = 19 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 remove_cap mds.0 on 0x10000000000.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 15 client.4638 remove_cap last one, closing snaprealm 0x7f9de400d920 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 20 client.4638 put_snap_realm 0x1 0x7f9de400d920 1 -> 0 2024-03-28T01:08:14.495 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 10 client.4638 _put_inode deleting 0x10000000000.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=1031/0 nlink=0 btime=2024-03-28T01:08:14.450279+0000 mtime=2024-03-28T01:08:14.461471+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=4 caps=- objectset[0x10000000000 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9de400dff0) 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 2 client.4638 _close_mds_session mds.0 seq 3 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 3) v7 -- 0x560b83faebe0 con 0x560b84057d30 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.476+0000 7f9e0324a480 2 client.4638 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(grant ino 0x10000000000 1 seq 4 caps=pAsLsXsFsxcrwb dirty=- wanted=- follows 0 size 1031/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:14.461471+0000 ctime 2024-03-28T01:08:14.466782+0000 change_attr 4) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df005f040 con 0x560b84057d30 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 10 client.4638 mds.0 seq now 4 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 4) v7 -- 0x7f9de400d700 con 0x560b84057d30 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 5 client.4638 handle_caps don't have vino 0x10000000000.head op 0, immediately releasing 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_cap_release(3) v2 -- 0x560b83faea30 con 0x560b84057d30 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, size was 0+0 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 20 client.4638 trim_cache size 0 max 16384 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.477+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, size still 0+0 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9de400d700 con 0x560b84057d30 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 10 client.4638 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:14.496 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 5 client.4638 _closed_mds_session mds.0 seq 4 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 1 -- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 msgr2=0x560b83f2b470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84057d30 0x560b83f2b470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 10 client.4638 remove_session_caps mds.0 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 10 client.4638 kick_requests_closed for mds.0 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, size was 0+0 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 20 client.4638 trim_cache size 0 max 16384 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9dfca00640 10 client.4638 unmounting: trim pass, size still 0+0 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9e0324a480 2 client.4638 unmounted. 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9e0324a480 1 client.4638 shutdown 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x560b83fab840 msgr2=0x560b83fadcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.481+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x560b83fab840 0x560b83fadcd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 msgr2=0x560b83f2c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2977342527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b84050250 0x560b83f2c980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9df800db30 tx=0x7f9df8001660 comp rx=0 tx=0).stop 2024-03-28T01:08:14.497 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e00600640 1 -- 172.21.15.33:0/2977342527 reap_dead start 2024-03-28T01:08:14.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 shutdown_connections 2024-03-28T01:08:14.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 >> 172.21.15.33:0/2977342527 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:14.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 shutdown_connections 2024-03-28T01:08:14.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e0324a480 1 -- 172.21.15.33:0/2977342527 wait complete. 2024-03-28T01:08:14.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.482+0000 7f9e0324a480 20 client.4638 trim_cache size 0 max 16384 2024-03-28T01:08:14.498 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritev (1055 ms) 2024-03-28T01:08:14.498 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritevNullContext 2024-03-28T01:08:14.503 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.484+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:14.503 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.485+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:14.503 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.486+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.503 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.486+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 0x560b83fb3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.503 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.486+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84055040 0x560b83fbb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.503 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.486+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4007b00 con 0x560b83fb4c30 2024-03-28T01:08:14.503 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.486+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8404fef0 con 0x560b8404fa40 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.486+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbb5b0 con 0x560b84055040 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83fb8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49422/0 (socket says 172.21.15.33:49422) 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 -- 172.21.15.33:0/3957288799 learned_addr learned my addr 172.21.15.33:0/3957288799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9e00600640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84055040 0x560b83fbb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dff200640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 0x560b83fb3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 -- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84055040 msgr2=0x560b83fbb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84055040 0x560b83fbb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 -- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 msgr2=0x560b83fb3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 0x560b83fb3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.487+0000 7f9dffc00640 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f281d0 con 0x560b83fb4c30 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.488+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83fb8730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9df001bd70 tx=0x7f9df004c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88b80ef9b519466 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.488+0000 7f9e0324a480 10 client.4641 Subscribing to map 'mdsmap' 2024-03-28T01:08:14.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.488+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mdsmap=0+}) v3 -- 0x560b83f28570 con 0x560b83fb4c30 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.488+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0050040 con 0x560b83fb4c30 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.488+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df0005ad0 con 0x560b83fb4c30 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.488+0000 7f9df5600640 20 client.4641 tick 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9df5600640 20 client.4641 collect_and_send_metrics 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9df5600640 20 client.4641 collect_and_send_global_metrics 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9df5600640 5 client.4641 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9df5600640 20 client.4641 trim_cache size 0 max 16384 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9df5600640 20 client.4641 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0017dd0 con 0x560b83fb4c30 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9dfde00640 1 client.4641 handle_mds_map epoch 8 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9e0324a480 20 client.4641 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9e0324a480 10 client.4641 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:14.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9e0324a480 20 client.4641 mds is 0 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9e0324a480 10 client.4641 _open_mds_session mds.0 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f2e110 0x560b83f2b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f2b640 con 0x560b83f2e110 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9e0324a480 10 client.4641 waiting for session to mds.0 to open 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.489+0000 7f9dff200640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f2e110 0x560b83f2b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.490+0000 7f9dff200640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f2e110 0x560b83f2b100 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291f6389f303115f server_cookie=bcff850362f00c91 in_seq=0 out_seq=0 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.490+0000 7f9dfde00640 10 client.4641 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f2b640 con 0x560b83f2e110 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9dfde00640 10 client.4641 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9dfde00640 10 client.4641 renew_caps mds.0 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc009b50 con 0x560b83f2e110 2024-03-28T01:08:14.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9dfde00640 10 client.4641 connect_mds_targets for mds.0 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 10 client.4641 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 20 client.4641 mds is 0 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 10 client.4641 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 20 client.4641 encode_cap_releases enter (req: 0x560b83f2dd50, mds: 0) 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 20 client.4641 send_request set sent_stamp to 2024-03-28T01:08:14.494451+0000 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 10 client.4641 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:14.489856+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:14.489856+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84059880 con 0x560b83f2e110 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.493+0000 7f9e0324a480 20 client.4641 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc009b50 con 0x560b83f2e110 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de8018040 con 0x560b83f2e110 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:14.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 insert_trace from 2024-03-28T01:08:14.494451+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 features 0xffffffffffffffff 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 update_snap_trace len 70 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 get_snap_realm 0x1 0x7f9dcc009d70, nref now is 1 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 15 client.4641 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 15 client.4641 snapc 1=[] 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 put_snap_realm 0x1 0x7f9dcc009d70 2 -> 1 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 hrm is_target=1 is_dentry=0 2024-03-28T01:08:14.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc009e60) - ctime 2024-03-28T01:08:14.466782+0000 mtime 2024-03-28T01:08:14.466782+0000 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 dir hash is 2 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 12 client.4641 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 2 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 12 client.4641 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=- has_dir_layout 0x7f9dcc009e60) caps pAsLsXsFs 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 get_snap_realm 0x1 0x7f9dcc009d70, nref was 1 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 get_snap_realm 0x1 0x7f9dcc009d70, nref now is 2 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 15 client.4641 add_update_cap first one, opened snaprealm 0x7f9dcc009d70 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc009e60) 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 add_update_cap calling signal_caps_inode 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc009e60) 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 put_snap_realm 0x1 0x7f9dcc009d70 2 -> 1 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:14.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 lat 0.000833 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 _ll_get 0x7f9dcc009e60 0x1 -> 1 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 2 client.4641 unmount 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 2 client.4641 unmounting 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b84059bd0 con 0x560b83f2e110 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 10 client.4641 _ll_drop_pins 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 _ll_put 0x7f9dcc009e60 0x1 1 -> 0 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 10 client.4641 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) 2024-03-28T01:08:14.510 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 10 client.4641 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 10 client.4641 nothing to flush 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 10 client.4641 flush_caps_sync 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 10 client.4641 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 20 client.4641 trim_cache size 0 max 16384 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 10 client.4641 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 6 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 2 client.4641 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 1 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 unmounting: trim pass, size was 0+1 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 20 client.4641 trim_cache size 0 max 16384 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9dfde00640 10 client.4641 unmounting: trim pass, size still 0+1 2024-03-28T01:08:14.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:14.494+0000 7f9e0324a480 2 client.4641 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:14.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:08:14.741 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:08:15.492 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 20 client.4641 tick 2024-03-28T01:08:15.492 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 10 client.4641 renew_caps() 2024-03-28T01:08:15.492 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 15 client.4641 renew_caps requesting from mds.0 2024-03-28T01:08:15.492 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 10 client.4641 renew_caps mds.0 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 2) v7 -- 0x7f9de400d700 con 0x560b83f2e110 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 20 client.4641 collect_and_send_metrics 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 20 client.4641 collect_and_send_global_metrics 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_metrics [client_metric_type: READ_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: WRITE_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: METADATA_LATENCY latency: 0.000833, avg_latency: 0.000833, sq_sum: 0, count=1][client_metric_type: CAP_INFO cap_hits: 0 cap_misses: 0 num_caps: 0][client_metric_type: DENTRY_LEASE dlease_hits: 0 dlease_misses: 0 num_dentries: 0][client_metric_type: OPENED_FILES opened_files: 0 total_inodes: 1][client_metric_type: PINNED_ICAPS pinned_icaps: 1 total_inodes: 1][client_metric_type: OPENED_INODES opened_inodes: 0 total_inodes: 1][client_metric_type: READ_IO_SIZES total_ops: 0 total_size: 0][client_metric_type: WRITE_IO_SIZES total_ops: 0 total_size: 0] v1 -- 0x7f9de4009f30 con 0x560b83f2e110 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 10 client.4641 _put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) n = 2 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 10 client.4641 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc009e60) 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 15 client.4641 remove_cap last one, closing snaprealm 0x7f9dcc009d70 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 20 client.4641 put_snap_realm 0x1 0x7f9dcc009d70 1 -> 0 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 10 client.4641 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=- COMPLETE has_dir_layout 0x7f9dcc009e60) 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 20 client.4641 trim_cache size 0 max 16384 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9df5600640 20 client.4641 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9e0324a480 2 client.4641 _close_mds_session mds.0 seq 0 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b84059e20 con 0x560b83f2e110 2024-03-28T01:08:15.493 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.489+0000 7f9e0324a480 2 client.4641 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:15.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.490+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (crc 0 0 0) 0x560b84059bd0 con 0x560b83f2e110 2024-03-28T01:08:15.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.490+0000 7f9dfde00640 10 client.4641 handle_client_session client_session(renewcaps seq 2) v1 from mds.0 2024-03-28T01:08:15.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.490+0000 7f9dfde00640 10 client.4641 unmounting: trim pass, size was 0+0 2024-03-28T01:08:15.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.490+0000 7f9dfde00640 20 client.4641 trim_cache size 0 max 16384 2024-03-28T01:08:15.494 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.490+0000 7f9dfde00640 10 client.4641 unmounting: trim pass, size still 0+0 2024-03-28T01:08:15.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.495+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b84059e20 con 0x560b83f2e110 2024-03-28T01:08:15.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 10 client.4641 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:15.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 5 client.4641 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:15.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 1 -- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f2e110 msgr2=0x560b83f2b100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:15.498 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f2e110 0x560b83f2b100 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.499 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 10 client.4641 remove_session_caps mds.0 2024-03-28T01:08:15.499 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 10 client.4641 kick_requests_closed for mds.0 2024-03-28T01:08:15.499 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 10 client.4641 unmounting: trim pass, size was 0+0 2024-03-28T01:08:15.499 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 20 client.4641 trim_cache size 0 max 16384 2024-03-28T01:08:15.499 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9dfde00640 10 client.4641 unmounting: trim pass, size still 0+0 2024-03-28T01:08:15.499 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 2 client.4641 unmounted. 2024-03-28T01:08:15.499 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 client.4641 shutdown 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 msgr2=0x560b83fb8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83fb8730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9df001bd70 tx=0x7f9df004c070 comp rx=0 tx=0).stop 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 shutdown_connections 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f2e110 0x560b83f2b100 unknown :-1 s=CLOSED pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84055040 0x560b83fbb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 0x560b83fb3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3957288799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83fb8730 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.500 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.496+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 >> 172.21.15.33:0/3957288799 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:15.501 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.497+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 shutdown_connections 2024-03-28T01:08:15.501 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.497+0000 7f9e0324a480 1 -- 172.21.15.33:0/3957288799 wait complete. 2024-03-28T01:08:15.501 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.497+0000 7f9e0324a480 20 client.4641 trim_cache size 0 max 16384 2024-03-28T01:08:15.501 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.498+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:15.502 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.500+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:15.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.501+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83f2e110 0x560b83fba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.501+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83f2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.501+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 0x560b8405a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.501+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0084a0 con 0x560b83fb4c30 2024-03-28T01:08:15.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.501+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbeab0 con 0x560b8404fa40 2024-03-28T01:08:15.504 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.501+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbec60 con 0x560b83f2e110 2024-03-28T01:08:15.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83f2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83f2cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49424/0 (socket says 172.21.15.33:49424) 2024-03-28T01:08:15.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 -- 172.21.15.33:0/4171161665 learned_addr learned my addr 172.21.15.33:0/4171161665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:15.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83f2e110 0x560b83fba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.505 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9e00600640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 0x560b8405a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 -- 172.21.15.33:0/4171161665 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83f2e110 msgr2=0x560b83fba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83f2e110 0x560b83fba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 -- 172.21.15.33:0/4171161665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 msgr2=0x560b8405a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8404fa40 0x560b8405a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.502+0000 7f9dff200640 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b840523c0 con 0x560b83fb4c30 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9e0324a480 10 client.4644 Subscribing to map 'mdsmap' 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mdsmap=0+}) v3 -- 0x560b84052760 con 0x560b83fb4c30 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9dff200640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83f2cbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9de800f5a0 tx=0x7f9de80019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e653271f224771 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:15.506 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9dfca00640 10 client.4644 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:15.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9df6000640 20 client.4644 tick 2024-03-28T01:08:15.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9df6000640 20 client.4644 collect_and_send_metrics 2024-03-28T01:08:15.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9df6000640 20 client.4644 collect_and_send_global_metrics 2024-03-28T01:08:15.507 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9df6000640 5 client.4644 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9df6000640 20 client.4644 trim_cache size 0 max 16384 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9df6000640 20 client.4644 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.503+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc002670 con 0x560b83fb4c30 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de802bb40 con 0x560b83fb4c30 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025070 con 0x560b83fb4c30 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9dfca00640 1 client.4644 handle_mds_map epoch 8 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9e0324a480 20 client.4644 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9e0324a480 10 client.4644 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9e0324a480 20 client.4644 mds is 0 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9e0324a480 10 client.4644 _open_mds_session mds.0 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84055040 0x560b83f2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83fb41d0 con 0x560b84055040 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9e0324a480 10 client.4644 waiting for session to mds.0 to open 2024-03-28T01:08:15.508 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.504+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84055040 0x560b83f2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.505+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84055040 0x560b83f2b120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17be3ddd36a69489 server_cookie=592425a743b8faf4 in_seq=0 out_seq=0 2024-03-28T01:08:15.509 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.505+0000 7f9dfca00640 10 client.4644 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:15.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83fb41d0 con 0x560b84055040 2024-03-28T01:08:15.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9dfca00640 10 client.4644 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:15.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9dfca00640 10 client.4644 renew_caps mds.0 2024-03-28T01:08:15.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc00a150 con 0x560b84055040 2024-03-28T01:08:15.511 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9dfca00640 10 client.4644 connect_mds_targets for mds.0 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 10 client.4644 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 20 client.4644 mds is 0 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 10 client.4644 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 20 client.4644 encode_cap_releases enter (req: 0x560b83fbb3f0, mds: 0) 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 20 client.4644 send_request set sent_stamp to 2024-03-28T01:08:15.509535+0000 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 10 client.4644 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:15.504913+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:15.504913+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8406aab0 con 0x560b84055040 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.508+0000 7f9e0324a480 20 client.4644 awaiting reply|forward|kick on 0x7ffe68269860 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc00a150 con 0x560b84055040 2024-03-28T01:08:15.512 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 10 client.4644 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059030 con 0x560b84055040 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 20 client.4644 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 10 client.4644 insert_trace from 2024-03-28T01:08:15.509535+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 10 client.4644 features 0xffffffffffffffff 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 10 client.4644 update_snap_trace len 70 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref now is 1 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 10 client.4644 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 10 client.4644 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 15 client.4644 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.509+0000 7f9dfca00640 15 client.4644 snapc 1=[] 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 10 client.4644 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:15.514 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 put_snap_realm 0x1 0x7f9dcc00a370 2 -> 1 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 10 client.4644 hrm is_target=1 is_dentry=0 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 10 client.4644 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00a460) - ctime 2024-03-28T01:08:14.466782+0000 mtime 2024-03-28T01:08:14.466782+0000 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 dir hash is 2 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 12 client.4644 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 2 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 12 client.4644 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=- has_dir_layout 0x7f9dcc00a460) caps pAsLsXsFs 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref was 1 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref now is 2 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 15 client.4644 add_update_cap first one, opened snaprealm 0x7f9dcc00a370 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 10 client.4644 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 10 client.4644 add_update_cap calling signal_caps_inode 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 10 client.4644 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 put_snap_realm 0x1 0x7f9dcc00a370 2 -> 1 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 handle_client_reply signalling caller 0x7ffe68269860 2024-03-28T01:08:15.515 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 lat 0.001221 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 _ll_get 0x7f9dcc00a460 0x1 -> 1 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 8 client.4644 _ll_create 0x1.head test_llreadvllwritevnullcontextfile44790 0666 578, uid 0, gid 0 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 10 client.4644 _lookup concluded ENOENT locally for 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460) dn 'test_llreadvllwritevnullcontextfile44790' 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 may_create 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 10 client.4644 _getattr mask As issued=1 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 3 client.4644 may_create 0x7f9dcc00a460 = 0 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 8 client.4644 _create(0x1 test_llreadvllwritevnullcontextfile44790, 0666) 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 get_or_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460) name test_llreadvllwritevnullcontextfile44790 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 15 open_dir 0x560b83fb4690 on 0x7f9dcc00a460 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 15 client.4644 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460) 'test_llreadvllwritevnullcontextfile44790' dn 0x1.head["test_llreadvllwritevnullcontextfile44790"] (new dn) 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 choose_target_mds inode dir hash is 2 on test_llreadvllwritevnullcontextfile44790 => 3778639343 2024-03-28T01:08:15.516 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 choose_target_mds 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460) is_hash=1 hash=3778639343 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 10 client.4644 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 mds is 0 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 10 client.4644 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 encode_cap_releases enter (req: 0x560b8406ade0, mds: 0) 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 encode_dentry_release enter(dn:0x560b83eac9f0) 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a460), req:0x560b8406ade0 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 send_request set sent_stamp to 2024-03-28T01:08:15.510933+0000 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 10 client.4644 send_request client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevnullcontextfile44790 2024-03-28T01:08:15.510854+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevnullcontextfile44790 2024-03-28T01:08:15.510854+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8406b300 con 0x560b84055040 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9e0324a480 20 client.4644 awaiting reply|forward|kick on 0x7ffe682696a0 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.517 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.510+0000 7f9dfca00640 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.518 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 861+0+0 (crc 0 0 0) 0x7f9df0005ad0 con 0x560b84055040 2024-03-28T01:08:15.518 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:15.518 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 insert_trace from 2024-03-28T01:08:15.510933+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:15.518 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 features 0xffffffffffffffff 2024-03-28T01:08:15.518 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 update_snap_trace len 70 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref was 1 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref now is 2 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 hrm is_target=1 is_dentry=1 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 update_inode_file_time 0x10000000001.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) - ctime 2024-03-28T01:08:15.510854+0000 mtime 2024-03-28T01:08:15.510854+0000 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 12 client.4644 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 12 client.4644 add_update_inode adding 0x10000000001.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=- objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref was 2 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref now is 3 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 15 client.4644 add_update_cap first one, opened snaprealm 0x7f9dcc00a370 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000001.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 add_update_cap calling signal_caps_inode 2024-03-28T01:08:15.519 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 12 client.4644 add_update_inode had 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a460) caps pAsLsXsFs 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 update_inode_file_time 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:14.466782+0000 ctime=2024-03-28T01:08:14.466782+0000 change_attr=2 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a460) pAsLsXs ctime 2024-03-28T01:08:15.510854+0000 mtime 2024-03-28T01:08:15.510854+0000 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 dir hash is 2 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 12 client.4644 add_update_inode client inode change_attr: 2 , mds inodestat change_attr: 3 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 clearing (I_COMPLETE|I_DIR_ORDERED) on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 10 client.4644 add_update_cap calling signal_caps_inode 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 12 client.4644 insert_dentry_inode 'test_llreadvllwritevnullcontextfile44790' vino 0x10000000001.head in dir 0x1.head dn 0x560b83eac9f0 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 15 client.4644 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) 'test_llreadvllwritevnullcontextfile44790' to inode 0x7f9dcc00aa40 dn 0x1.head["test_llreadvllwritevnullcontextfile44790"] (old dn) 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 link inode 0x7f9dcc00aa40 parents now 0x1.head["test_llreadvllwritevnullcontextfile44790"] 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 15 client.4644 update_dentry_lease 0x1.head["test_llreadvllwritevnullcontextfile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:15.510933+0000 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 put_snap_realm 0x1 0x7f9dcc00a370 3 -> 2 2024-03-28T01:08:15.520 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 handle_client_reply signalling caller 0x7ffe682696a0 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 client.4644 delegated_inos: [] 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 client.4644 make_request created ino 0x10000000001 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 make_request target is 0x10000000001.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 lat 0.005426 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000001.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000001.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 client.4644 _create_fh 0x10000000001 mode 3 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 trim_cache size 1 max 16384 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 8 client.4644 create(#0x1/test_llreadvllwritevnullcontextfile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 _ll_create created = 1 2024-03-28T01:08:15.521 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 8 client.4644 _ll_create 0x1.head test_llreadvllwritevnullcontextfile44790 0666 578 = 0 (0x560b8406b730 10000000001) 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 _ll_get 0x7f9dcc00aa40 0x10000000001 -> 1 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 client.4644 fill_statx on 0x10000000001 snap/devhead mode 0100666 mtime 2024-03-28T01:08:15.510854+0000 ctime 2024-03-28T01:08:15.510854+0000 change_attr 0 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 client.4644 get_quota_root realm 0x1 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 client.4644 get_quota_root 0x10000000001.head -> 0x1.head 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 10 client.4644 cur file size is 0 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fa77b0 0x560b83fa9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- osd_op(unknown.0.0:1 3.7 3:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x560b83faa1b0 con 0x560b83fa77b0 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- osd_op(unknown.0.0:2 3.7 3:e2e07930:::10000000001.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x560b83faaa20 con 0x560b83fa77b0 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.515+0000 7f9dfca00640 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.522 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.516+0000 7f9e00600640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fa77b0 0x560b83fa9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.524 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.517+0000 7f9e00600640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fa77b0 0x560b83fa9c70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=a378d57c162edeb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:15.524 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.517+0000 7f9dfca00640 10 client.4644 ms_handle_connect on v2:172.21.15.33:6800/2717396394 2024-03-28T01:08:15.524 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.517+0000 7f9e00600640 1 -- 172.21.15.33:0/4171161665 <== osd.3 v2:172.21.15.33:6800/2717396394 1 ==== osd_op_reply(1 10000000001.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8029040 con 0x560b83fa77b0 2024-03-28T01:08:15.524 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e00600640 1 -- 172.21.15.33:0/4171161665 <== osd.3 v2:172.21.15.33:6800/2717396394 2 ==== osd_op_reply(2 10000000001.00000000 [create] v22'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8029040 con 0x560b83fa77b0 2024-03-28T01:08:15.524 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 wanted_max_size 0 -> 14 2024-03-28T01:08:15.524 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 get_caps 0x10000000001.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:15.524 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 20 client.4644 clear_suid_sgid 0x10000000001.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={4=0,4096=1} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 5 client.4644 get_cap_ref got first FILE_BUFFER ref on 0x10000000001.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={4=0,4096=1,8192=0} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 5 client.4644 get_cap_ref got first FILE_CACHE ref on 0x10000000001.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={4=0,1024=0,4096=1,8192=0} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000001.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) - -> Fw 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 cap_delay_requeue on 0x10000000001.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 get_quota_root realm 0x1 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 get_quota_root 0x10000000001.head -> 0x1.head 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 7 client.4644 wrote to 14, extending file size 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000001.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) Fw -> Fw 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 cap_delay_requeue on 0x10000000001.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=1,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 3 client.4644 pwritev(0x560b8406b730, "...", 14, 0) = 14 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 get_caps 0x10000000001.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,4096=0,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) have pAsxLsXsxFsxcrwb need Fr want Fc revoking - 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 _read_async 0x10000000001.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=1,4096=0,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 0~14 2024-03-28T01:08:15.525 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 10 client.4644 min_bytes=131072 max_bytes=16777216 max_periods=4 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 3 client.4644 preadv(0x560b8406b730, 0) = 14 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.520+0000 7f9e0324a480 3 client.4644 ll_release (fh)0x560b8406b730 0x10000000001 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 8 client.4644 _release_fh 0x560b8406b730 mode 3 on 0x10000000001.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=1} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 _flush 0x10000000001.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- osd_op(unknown.0.0:3 3.7 3:e2e07930:::10000000001.00000000:head [write 0~14 in=14b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x560b83fa6e60 con 0x560b83fa77b0 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 check_caps on 0x10000000001.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 delaying cap release 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 cap_delay_requeue on 0x10000000001.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 _release_fh 0x560b8406b730 on inode 0x10000000001.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) no async_err state 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 3 client.4644 ll_unlink 0x1.head test_llreadvllwritevnullcontextfile44790 2024-03-28T01:08:15.526 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 may_delete 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460); ; name test_llreadvllwritevnullcontextfile44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 _getattr mask As issued=1 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 3 client.4644 may_delete 0x7f9dcc00a460 = 0 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 8 client.4644 _unlink(0x1 test_llreadvllwritevnullcontextfile44790 uid 0 gid 0) 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 get_or_create 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) name test_llreadvllwritevnullcontextfile44790 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 _lookup have 0x1.head["test_llreadvllwritevnullcontextfile44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 _lookup 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) test_llreadvllwritevnullcontextfile44790 = 0x10000000001.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 choose_target_mds inode dir hash is 2 on test_llreadvllwritevnullcontextfile44790 => 3778639343 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 choose_target_mds 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460) is_hash=1 hash=3778639343 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 choose_target_mds from dirfragtree hash 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 mds is 0 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:15.527 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 encode_cap_releases enter (req: 0x560b8406be40, mds: 0) 2024-03-28T01:08:15.528 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 encode_inode_release enter(in:0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40), req:0x560b8406be40 mds:0, drop:LsxXx, unless:-, force:0) 2024-03-28T01:08:15.528 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 encode_dentry_release enter(dn:0x560b83eac9f0) 2024-03-28T01:08:15.528 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a460), req:0x560b8406be40 mds:0, drop:Fs, unless:Fx, force:1) 2024-03-28T01:08:15.528 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 send_request set sent_stamp to 2024-03-28T01:08:15.522316+0000 2024-03-28T01:08:15.528 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 10 client.4644 send_request client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevnullcontextfile44790 2024-03-28T01:08:15.522182+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:15.528 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevnullcontextfile44790 2024-03-28T01:08:15.522182+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8406c230 con 0x560b84055040 2024-03-28T01:08:15.528 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.521+0000 7f9e0324a480 20 client.4644 awaiting reply|forward|kick on 0x7ffe68269880 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_caps(revoke ino 0x10000000001 1 seq 2 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:15.510854+0000 ctime 2024-03-28T01:08:15.510854+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081070 con 0x560b84055040 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 5 client.4644 set_cap_epoch_barrier epoch = 20 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 mds.0 seq now 1 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 5 client.4644 handle_cap_grant on in 0x10000000001 mds.0 seq 2 caps now pAsXsFsxcrwb was pAsxXsFsxcrwb 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 update_inode_file_time 0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) pAsxXsFsxcrwb ctime 2024-03-28T01:08:15.510854+0000 mtime 2024-03-28T01:08:15.510854+0000 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 revocation of Ax 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 check_caps on 0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) wanted Fxcb used Fcb issued pAsXsFsxcrwb revoking Ax flags=1 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsFsxcrwb revoking Ax 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 completed revocation of Ax 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 mark_caps_flushing Fw 0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 mark_caps_clean 0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) 2024-03-28T01:08:15.530 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 send_cap 0x10000000001.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] parents=0x1.head["test_llreadvllwritevnullcontextfile44790"] 0x7f9dcc00aa40) mds.0 seq 2 used Fcb want Fxcb flush Fw retain pAsLsxXsxFsxcrwbl held pAsxXsFsxcrwb revoking Ax dropping - 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 15 client.4644 auth cap, requesting max_size 14 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000001 1 seq 2 tid 2 caps=pAsXsFsxcrwb dirty=Fw wanted=Fxcb follows 1 size 14/14 ts 1/18446744073709551615 mtime 2024-03-28T01:08:15.521533+0000 ctime 2024-03-28T01:08:15.521533+0000 change_attr 1) v12 -- 0x7f9dcc00b020 con 0x560b84055040 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.522+0000 7f9dfca00640 10 client.4644 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_reply(???:3 = 0 (0) Success unsafe) v1 ==== 509+0+0 (crc 0 0 0) 0x7f9df0019c40 con 0x560b84055040 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 handle_client_reply got a reply. Safe:0 tid 3 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 insert_trace from 2024-03-28T01:08:15.522316+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 features 0xffffffffffffffff 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 update_snap_trace len 70 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref was 2 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 get_snap_realm 0x1 0x7f9dcc00a370, nref now is 3 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 hrm is_target=0 is_dentry=1 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 12 client.4644 add_update_inode had 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) caps pAsLsXs 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 update_inode_file_time 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.510854+0000 ctime=2024-03-28T01:08:15.510854+0000 change_attr=3 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) pAsLsXs ctime 2024-03-28T01:08:15.522182+0000 mtime 2024-03-28T01:08:15.522182+0000 2024-03-28T01:08:15.531 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 dir hash is 2 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 12 client.4644 add_update_inode client inode change_attr: 3 , mds inodestat change_attr: 4 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 15 client.4644 unlink dir 0x7f9dcc00a460 'test_llreadvllwritevnullcontextfile44790' dn 0x560b83eac9f0 inode 0x7f9dcc00aa40 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 unlink inode 0x7f9dcc00aa40 parents now 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 put_snap_realm 0x1 0x7f9dcc00a370 3 -> 2 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 handle_client_reply signalling caller 0x7ffe68269880 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 20 client.4644 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 20 client.4644 lat 0.002807 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 20 client.4644 trim_cache size 1 max 16384 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 8 client.4644 unlink(#0x1/test_llreadvllwritevnullcontextfile44790) = 0 2024-03-28T01:08:15.532 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 14] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 2 client.4644 unmount 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 2 client.4644 unmounting 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b8406c630 con 0x560b84055040 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 10 client.4644 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size was 1+2 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 20 client.4644 trim_cache size 1 max 16384 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size still 1+2 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e0324a480 10 client.4644 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9e00600640 1 -- 172.21.15.33:0/4171161665 <== osd.3 v2:172.21.15.33:6800/2717396394 3 ==== osd_op_reply(3 10000000001.00000000 [write 0~14] v22'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8029040 con 0x560b83fa77b0 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9df7400640 10 client.4644 _flushed 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=1,2048=0,4096=0,8192=1} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9df7400640 5 client.4644 put_cap_ref dropped last FILE_BUFFER ref on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9df7400640 5 client.4644 put_cap_ref dropped last FILE_CACHE ref on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9df7400640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) n = 2 2024-03-28T01:08:15.533 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.524+0000 7f9df7400640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=1 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.535 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_caps(grant ino 0x10000000001 1 seq 3 caps=pAsLsXsFsxcrwb dirty=- wanted=LsFxcb follows 0 size 14/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:15.521533+0000 ctime 2024-03-28T01:08:15.522182+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dcc00b020 con 0x560b84055040 2024-03-28T01:08:15.535 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 mds.0 seq now 2 2024-03-28T01:08:15.535 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 5 client.4644 handle_cap_grant on in 0x10000000001 mds.0 seq 3 caps now pAsLsXsFsxcrwb was pAsXsFsxcrwb 2024-03-28T01:08:15.535 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 update_inode_file_time 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.521533+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) pAsXsFsxcrwb ctime 2024-03-28T01:08:15.522182+0000 mtime 2024-03-28T01:08:15.521533+0000 2024-03-28T01:08:15.535 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 grant, new caps are Ls 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 5 client.4644 _try_to_trim_inode in 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size was 1+2 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 20 client.4644 trim_cache size 1 max 16384 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size still 1+2 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df001ebf0 con 0x560b84055040 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 20 client.4644 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 insert_trace from 2024-03-28T01:08:15.522316+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size was 1+2 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 20 client.4644 trim_cache size 1 max 16384 2024-03-28T01:08:15.536 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 15 client.4644 trim_dentry unlinking dn test_llreadvllwritevnullcontextfile44790 in dir 0x1 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 15 client.4644 unlink dir 0x7f9dcc00a460 'test_llreadvllwritevnullcontextfile44790' dn 0x560b83eac9f0 inode 0 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 15 client.4644 unlink removing 'test_llreadvllwritevnullcontextfile44790' dn 0x560b83eac9f0 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 15 client.4644 close_dir dir 0x560b83fb4690 on 0x7f9dcc00a460 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 _ll_drop_pins 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 _ll_put 0x7f9dcc00aa40 0x10000000001 1 -> 0 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 _ll_put 0x7f9dcc00a460 0x1 2 -> 0 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.537 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 _release 0x10000000001.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 _invalidate_inode_cache 0x10000000001.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 _flush 0x10000000001.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 nothing to flush 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 _release 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 _flush 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 nothing to flush 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 20 client.4644 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 1 2024-03-28T01:08:15.538 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 flush_caps_sync 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 wait_sync_caps want 2 (last is 2, 1 total flushing) 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.529+0000 7f9e0324a480 10 client.4644 waiting on mds.0 tid 2 (want 2) 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_caps(flush_ack ino 0x10000000001 1 seq 2 tid 2 caps=pAsXsFsxcrwb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0054bb0 con 0x560b84055040 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 10 client.4644 mds.0 seq now 3 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 5 client.4644 handle_cap_flush_ack mds.0 cleaned Fw on 0x10000000001.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) with Fw 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 5 client.4644 flushing_caps Fw -> - 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 10 client.4644 0x10000000001.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) !flushing 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 10 client.4644 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 20 client.4644 put_inode on 0x10000000001.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) n = 1 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size was 0+2 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 20 client.4644 trim_cache size 0 max 16384 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size still 0+2 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 10 client.4644 wait_sync_caps want 2 (last is 2, 0 total flushing) 2024-03-28T01:08:15.539 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 20 client.4644 trim_cache size 0 max 16384 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 10 client.4644 _put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) n = 11 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 10 client.4644 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 15 client.4644 remove_cap last one, closing snaprealm 0x7f9dcc00a370 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 20 client.4644 put_snap_realm 0x1 0x7f9dcc00a370 2 -> 1 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 10 client.4644 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=- has_dir_layout 0x7f9dcc00a460) 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 10 client.4644 _put_inode on 0x10000000001.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) n = 18 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 10 client.4644 remove_cap mds.0 on 0x10000000001.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 15 client.4644 remove_cap last one, closing snaprealm 0x7f9dcc00a370 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 20 client.4644 put_snap_realm 0x1 0x7f9dcc00a370 1 -> 0 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 10 client.4644 _put_inode deleting 0x10000000001.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,2048=0,4096=0,8192=0} open={3=0} mode=100666 size=14/4194304 nlink=0 btime=2024-03-28T01:08:15.510854+0000 mtime=2024-03-28T01:08:15.521533+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=1 caps=- objectset[0x10000000001 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00aa40) 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 2 client.4644 _close_mds_session mds.0 seq 3 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 3) v7 -- 0x560b83faa720 con 0x560b84055040 2024-03-28T01:08:15.540 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.530+0000 7f9e0324a480 2 client.4644 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa720 con 0x560b84055040 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 10 client.4644 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 5 client.4644 _closed_mds_session mds.0 seq 3 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 1 -- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84055040 msgr2=0x560b83f2b120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84055040 0x560b83f2b120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 10 client.4644 remove_session_caps mds.0 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 10 client.4644 kick_requests_closed for mds.0 2024-03-28T01:08:15.542 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size was 0+0 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 20 client.4644 trim_cache size 0 max 16384 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9dfca00640 10 client.4644 unmounting: trim pass, size still 0+0 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 2 client.4644 unmounted. 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 1 client.4644 shutdown 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fa77b0 msgr2=0x560b83fa9c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fa77b0 0x560b83fa9c70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 msgr2=0x560b83f2cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4171161665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb4c30 0x560b83f2cbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9de800f5a0 tx=0x7f9de80019c0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e00600640 1 -- 172.21.15.33:0/4171161665 reap_dead start 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 shutdown_connections 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.534+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 >> 172.21.15.33:0/4171161665 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.535+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 shutdown_connections 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.535+0000 7f9e0324a480 1 -- 172.21.15.33:0/4171161665 wait complete. 2024-03-28T01:08:15.543 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.535+0000 7f9e0324a480 20 client.4644 trim_cache size 0 max 16384 2024-03-28T01:08:15.544 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritevNullContext (1052 ms) 2024-03-28T01:08:15.544 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritevOPathFileHandle 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.536+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b8405a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b83f2cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc004ed0 con 0x560b83eff1f0 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b84059bb0 con 0x560b8405bb20 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b84059d60 con 0x560b83ee5730 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b83f2cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b83f2cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49432/0 (socket says 172.21.15.33:49432) 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.537+0000 7f9e00600640 1 -- 172.21.15.33:0/1594790902 learned_addr learned my addr 172.21.15.33:0/1594790902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:15.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9dff200640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b8405a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9e00600640 1 -- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 msgr2=0x560b83fba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9e00600640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9e00600640 1 -- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 msgr2=0x560b8405a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9e00600640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b8405a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9e00600640 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b840618c0 con 0x560b83eff1f0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9dff200640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b8405a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9e00600640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b83f2cb50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001e00 tx=0x7f9df8003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4100fbc54b1fdf38 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.538+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df804c060 con 0x560b83eff1f0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df800cc40 con 0x560b83eff1f0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 10 client.4647 Subscribing to map 'mdsmap' 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83f27cf0 con 0x560b83eff1f0 2024-03-28T01:08:15.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9df5600640 20 client.4647 tick 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9df5600640 20 client.4647 collect_and_send_metrics 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9df5600640 20 client.4647 collect_and_send_global_metrics 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9df5600640 5 client.4647 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9df5600640 20 client.4647 trim_cache size 0 max 16384 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9df5600640 20 client.4647 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df800ce40 con 0x560b83eff1f0 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b83eff1f0 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9dfde00640 1 client.4647 handle_mds_map epoch 8 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 20 client.4647 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 10 client.4647 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 20 client.4647 mds is 0 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 10 client.4647 _open_mds_session mds.0 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f283a0 0x560b83f2b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b84061b20 con 0x560b83f283a0 2024-03-28T01:08:15.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.539+0000 7f9e0324a480 10 client.4647 waiting for session to mds.0 to open 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.540+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f283a0 0x560b83f2b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.540+0000 7f9dfde00640 10 client.4647 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.540+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f283a0 0x560b83f2b100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d76599d541348f7 server_cookie=6369e37bec06a91 in_seq=0 out_seq=0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b84061b20 con 0x560b83f283a0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9dfde00640 10 client.4647 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9dfde00640 10 client.4647 renew_caps mds.0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc00a4b0 con 0x560b83f283a0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9dfde00640 10 client.4647 connect_mds_targets for mds.0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 10 client.4647 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 20 client.4647 mds is 0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 10 client.4647 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 20 client.4647 encode_cap_releases enter (req: 0x560b84055250, mds: 0) 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 20 client.4647 send_request set sent_stamp to 2024-03-28T01:08:15.545378+0000 2024-03-28T01:08:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 10 client.4647 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:15.540515+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:15.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:15.540515+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f28780 con 0x560b83f283a0 2024-03-28T01:08:15.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9e0324a480 20 client.4647 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:15.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc00a4b0 con 0x560b83f283a0 2024-03-28T01:08:15.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.544+0000 7f9dfde00640 10 client.4647 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:15.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b83f283a0 2024-03-28T01:08:15.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:15.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 insert_trace from 2024-03-28T01:08:15.545378+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:15.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 features 0xffffffffffffffff 2024-03-28T01:08:15.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 update_snap_trace len 70 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 get_snap_realm 0x1 0x7f9dcc00a6d0, nref now is 1 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 15 client.4647 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 15 client.4647 snapc 1=[] 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 put_snap_realm 0x1 0x7f9dcc00a6d0 2 -> 1 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 hrm is_target=1 is_dentry=0 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00a7c0) - ctime 2024-03-28T01:08:15.522182+0000 mtime 2024-03-28T01:08:15.522182+0000 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 dir hash is 2 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 12 client.4647 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 4 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 12 client.4647 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=- has_dir_layout 0x7f9dcc00a7c0) caps pAsLsXsFs 2024-03-28T01:08:15.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 get_snap_realm 0x1 0x7f9dcc00a6d0, nref was 1 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 get_snap_realm 0x1 0x7f9dcc00a6d0, nref now is 2 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 15 client.4647 add_update_cap first one, opened snaprealm 0x7f9dcc00a6d0 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a7c0) 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 add_update_cap calling signal_caps_inode 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a7c0) 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 put_snap_realm 0x1 0x7f9dcc00a6d0 2 -> 1 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 lat 0.000651 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 _ll_get 0x7f9dcc00a7c0 0x1 -> 1 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 2 client.4647 unmount 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 2 client.4647 unmounting 2024-03-28T01:08:15.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83ef9790 con 0x560b83f283a0 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 10 client.4647 _ll_drop_pins 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 _ll_put 0x7f9dcc00a7c0 0x1 1 -> 0 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 10 client.4647 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 10 client.4647 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 10 client.4647 nothing to flush 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 10 client.4647 flush_caps_sync 2024-03-28T01:08:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 10 client.4647 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 20 client.4647 trim_cache size 0 max 16384 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 10 client.4647 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 6 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 2 client.4647 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 1 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 unmounting: trim pass, size was 0+1 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 20 client.4647 trim_cache size 0 max 16384 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9dfde00640 10 client.4647 unmounting: trim pass, size still 0+1 2024-03-28T01:08:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:15.545+0000 7f9e0324a480 2 client.4647 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:16.545 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 20 client.4647 tick 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 10 client.4647 renew_caps() 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 15 client.4647 renew_caps requesting from mds.0 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 10 client.4647 renew_caps mds.0 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 2) v7 -- 0x7f9dc4000c00 con 0x560b83f283a0 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 20 client.4647 collect_and_send_metrics 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 20 client.4647 collect_and_send_global_metrics 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_metrics [client_metric_type: READ_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: WRITE_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: METADATA_LATENCY latency: 0.000651, avg_latency: 0.000651, sq_sum: 0, count=1][client_metric_type: CAP_INFO cap_hits: 0 cap_misses: 0 num_caps: 0][client_metric_type: DENTRY_LEASE dlease_hits: 0 dlease_misses: 0 num_dentries: 0][client_metric_type: OPENED_FILES opened_files: 0 total_inodes: 1][client_metric_type: PINNED_ICAPS pinned_icaps: 1 total_inodes: 1][client_metric_type: OPENED_INODES opened_inodes: 0 total_inodes: 1][client_metric_type: READ_IO_SIZES total_ops: 0 total_size: 0][client_metric_type: WRITE_IO_SIZES total_ops: 0 total_size: 0] v1 -- 0x7f9dc4001490 con 0x560b83f283a0 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 10 client.4647 _put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) n = 2 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 10 client.4647 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a7c0) 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 15 client.4647 remove_cap last one, closing snaprealm 0x7f9dcc00a6d0 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 20 client.4647 put_snap_realm 0x1 0x7f9dcc00a6d0 1 -> 0 2024-03-28T01:08:16.546 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 10 client.4647 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=- COMPLETE has_dir_layout 0x7f9dcc00a7c0) 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 20 client.4647 trim_cache size 0 max 16384 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9df5600640 20 client.4647 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9e0324a480 2 client.4647 _close_mds_session mds.0 seq 0 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.539+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83f2b9b0 con 0x560b83f283a0 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.540+0000 7f9e0324a480 2 client.4647 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.540+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83ef9790 con 0x560b83f283a0 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.540+0000 7f9dfde00640 10 client.4647 handle_client_session client_session(renewcaps seq 2) v1 from mds.0 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.540+0000 7f9dfde00640 10 client.4647 unmounting: trim pass, size was 0+0 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.540+0000 7f9dfde00640 20 client.4647 trim_cache size 0 max 16384 2024-03-28T01:08:16.547 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.540+0000 7f9dfde00640 10 client.4647 unmounting: trim pass, size still 0+0 2024-03-28T01:08:16.548 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83f2b9b0 con 0x560b83f283a0 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 10 client.4647 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 5 client.4647 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 1 -- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f283a0 msgr2=0x560b83f2b100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f283a0 0x560b83f2b100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 10 client.4647 remove_session_caps mds.0 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 10 client.4647 kick_requests_closed for mds.0 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 10 client.4647 unmounting: trim pass, size was 0+0 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 20 client.4647 trim_cache size 0 max 16384 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9dfde00640 10 client.4647 unmounting: trim pass, size still 0+0 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9e0324a480 2 client.4647 unmounted. 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9e0324a480 1 client.4647 shutdown 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 msgr2=0x560b83f2cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:16.549 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.546+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b83f2cb50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001e00 tx=0x7f9df8003850 comp rx=0 tx=0).stop 2024-03-28T01:08:16.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 shutdown_connections 2024-03-28T01:08:16.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83f283a0 0x560b83f2b100 unknown :-1 s=CLOSED pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.550 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b83f2cb50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b8405a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1594790902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 >> 172.21.15.33:0/1594790902 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 shutdown_connections 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 1 -- 172.21.15.33:0/1594790902 wait complete. 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.547+0000 7f9e0324a480 20 client.4647 trim_cache size 0 max 16384 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.548+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:16.551 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.549+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:16.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.549+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83ee5730 0x560b8405a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.549+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0072b0 con 0x560b83f283a0 2024-03-28T01:08:16.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f27e00 con 0x560b83ee5730 2024-03-28T01:08:16.552 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83f27fb0 con 0x560b83eff1f0 2024-03-28T01:08:16.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b83fb3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49442/0 (socket says 172.21.15.33:49442) 2024-03-28T01:08:16.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9e00600640 1 -- 172.21.15.33:0/3401211103 learned_addr learned my addr 172.21.15.33:0/3401211103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:16.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83ee5730 0x560b8405a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.553 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.550+0000 7f9dff200640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.551+0000 7f9dffc00640 1 -- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.554 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.551+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.551+0000 7f9dffc00640 1 -- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.551+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.551+0000 7f9dffc00640 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83fb1870 con 0x560b83ee5730 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.551+0000 7f9e00600640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83ee5730 0x560b8405a090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005e30 tx=0x7f9df0079000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25df907ed9912a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4001490 con 0x560b83ee5730 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9e0324a480 10 client.4555 Subscribing to map 'mdsmap' 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83fad220 con 0x560b83ee5730 2024-03-28T01:08:16.555 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df0002030 con 0x560b83ee5730 2024-03-28T01:08:16.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9df6000640 20 client.4555 tick 2024-03-28T01:08:16.556 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9df6000640 20 client.4555 collect_and_send_metrics 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9df6000640 20 client.4555 collect_and_send_global_metrics 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9df6000640 5 client.4555 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9df6000640 20 client.4555 trim_cache size 0 max 16384 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.552+0000 7f9df6000640 20 client.4555 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mon.1 v2:172.21.15.194:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b83faea30 con 0x560b83ee5730 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0091070 con 0x560b83ee5730 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9dfca00640 1 client.4555 handle_mds_map epoch 8 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9e0324a480 20 client.4555 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9e0324a480 10 client.4555 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9e0324a480 20 client.4555 mds is 0 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9e0324a480 10 client.4555 _open_mds_session mds.0 2024-03-28T01:08:16.557 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.558 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f2b9b0 con 0x560b8405bb20 2024-03-28T01:08:16.558 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9e0324a480 10 client.4555 waiting for session to mds.0 to open 2024-03-28T01:08:16.558 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.553+0000 7f9dff200640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.558 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.554+0000 7f9dff200640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2ca00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13edbdf1eaec140c server_cookie=9eddce49ecafede4 in_seq=0 out_seq=0 2024-03-28T01:08:16.558 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.554+0000 7f9dfca00640 10 client.4555 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f2b9b0 con 0x560b8405bb20 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9dfca00640 10 client.4555 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9dfca00640 10 client.4555 renew_caps mds.0 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc00a500 con 0x560b8405bb20 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9dfca00640 10 client.4555 connect_mds_targets for mds.0 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 10 client.4555 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 20 client.4555 mds is 0 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 10 client.4555 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 20 client.4555 encode_cap_releases enter (req: 0x560b83fae4f0, mds: 0) 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 20 client.4555 send_request set sent_stamp to 2024-03-28T01:08:16.559244+0000 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 10 client.4555 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.553980+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.561 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.553980+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2df50 con 0x560b8405bb20 2024-03-28T01:08:16.562 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9e0324a480 20 client.4555 awaiting reply|forward|kick on 0x7ffe68269840 2024-03-28T01:08:16.562 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc00a500 con 0x560b8405bb20 2024-03-28T01:08:16.562 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.558+0000 7f9dfca00640 10 client.4555 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de40033d0 con 0x560b8405bb20 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 insert_trace from 2024-03-28T01:08:16.559244+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 features 0xffffffffffffffff 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 update_snap_trace len 70 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref now is 1 2024-03-28T01:08:16.563 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 15 client.4555 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 15 client.4555 snapc 1=[] 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 put_snap_realm 0x1 0x7f9dcc00a720 2 -> 1 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 hrm is_target=1 is_dentry=0 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00a810) - ctime 2024-03-28T01:08:15.522182+0000 mtime 2024-03-28T01:08:15.522182+0000 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 dir hash is 2 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 12 client.4555 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 4 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 12 client.4555 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=- has_dir_layout 0x7f9dcc00a810) caps pAsLsXsFs 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref was 1 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref now is 2 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 15 client.4555 add_update_cap first one, opened snaprealm 0x7f9dcc00a720 2024-03-28T01:08:16.564 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 10 client.4555 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 put_snap_realm 0x1 0x7f9dcc00a720 2 -> 1 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 handle_client_reply signalling caller 0x7ffe68269840 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 lat 0.000754 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 _ll_get 0x7f9dcc00a810 0x1 -> 1 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 8 client.4555 _ll_create 0x1.head test_llreadvllwritevopathfilehandlefile44790 0666 66, uid 0, gid 0 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 10 client.4555 _lookup concluded ENOENT locally for 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810) dn 'test_llreadvllwritevopathfilehandlefile44790' 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 may_create 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 10 client.4555 _getattr mask As issued=1 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 3 client.4555 may_create 0x7f9dcc00a810 = 0 2024-03-28T01:08:16.565 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 8 client.4555 _create(0x1 test_llreadvllwritevopathfilehandlefile44790, 0666) 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 get_or_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810) name test_llreadvllwritevopathfilehandlefile44790 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 15 open_dir 0x560b83f2ddc0 on 0x7f9dcc00a810 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 15 client.4555 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810) 'test_llreadvllwritevopathfilehandlefile44790' dn 0x1.head["test_llreadvllwritevopathfilehandlefile44790"] (new dn) 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 choose_target_mds inode dir hash is 2 on test_llreadvllwritevopathfilehandlefile44790 => 3076465480 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 choose_target_mds 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810) is_hash=1 hash=3076465480 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 10 client.4555 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 mds is 0 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 10 client.4555 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 encode_cap_releases enter (req: 0x560b83fba370, mds: 0) 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 encode_dentry_release enter(dn:0x7f9de400d920) 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00a810), req:0x560b83fba370 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 send_request set sent_stamp to 2024-03-28T01:08:16.560108+0000 2024-03-28T01:08:16.566 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 10 client.4555 send_request client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevopathfilehandlefile44790 2024-03-28T01:08:16.560070+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.567 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevopathfilehandlefile44790 2024-03-28T01:08:16.560070+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fba860 con 0x560b8405bb20 2024-03-28T01:08:16.567 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9e0324a480 20 client.4555 awaiting reply|forward|kick on 0x7ffe68269680 2024-03-28T01:08:16.567 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.567 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.559+0000 7f9dfca00640 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.568 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 865+0+0 (crc 0 0 0) 0x7f9de8026040 con 0x560b8405bb20 2024-03-28T01:08:16.568 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 insert_trace from 2024-03-28T01:08:16.560108+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 features 0xffffffffffffffff 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 update_snap_trace len 70 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref was 1 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref now is 2 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 hrm is_target=1 is_dentry=1 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 update_inode_file_time 0x10000000002.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) - ctime 2024-03-28T01:08:16.560070+0000 mtime 2024-03-28T01:08:16.560070+0000 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 12 client.4555 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 12 client.4555 add_update_inode adding 0x10000000002.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=- objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref was 2 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref now is 3 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 15 client.4555 add_update_cap first one, opened snaprealm 0x7f9dcc00a720 2024-03-28T01:08:16.569 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000002.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 12 client.4555 add_update_inode had 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a810) caps pAsLsXsFs 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 update_inode_file_time 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:15.522182+0000 ctime=2024-03-28T01:08:15.522182+0000 change_attr=4 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a810) pAsLsXs ctime 2024-03-28T01:08:16.560070+0000 mtime 2024-03-28T01:08:16.560070+0000 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 dir hash is 2 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 12 client.4555 add_update_inode client inode change_attr: 4 , mds inodestat change_attr: 5 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 clearing (I_COMPLETE|I_DIR_ORDERED) on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 10 client.4555 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 12 client.4555 insert_dentry_inode 'test_llreadvllwritevopathfilehandlefile44790' vino 0x10000000002.head in dir 0x1.head dn 0x7f9de400d920 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 15 client.4555 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) 'test_llreadvllwritevopathfilehandlefile44790' to inode 0x7f9dcc00adf0 dn 0x1.head["test_llreadvllwritevopathfilehandlefile44790"] (old dn) 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 link inode 0x7f9dcc00adf0 parents now 0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 2024-03-28T01:08:16.570 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 15 client.4555 update_dentry_lease 0x1.head["test_llreadvllwritevopathfilehandlefile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:16.560108+0000 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 put_snap_realm 0x1 0x7f9dcc00a720 3 -> 2 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 handle_client_reply signalling caller 0x7ffe68269680 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9dfca00640 20 client.4555 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 20 client.4555 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 10 client.4555 delegated_inos: [] 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 10 client.4555 make_request created ino 0x10000000002 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 20 client.4555 make_request target is 0x10000000002.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 20 client.4555 lat 0.005526 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000002.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000002.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.564+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 _create_fh 0x10000000002 mode 3 2024-03-28T01:08:16.571 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 trim_cache size 1 max 16384 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 8 client.4555 create(#0x1/test_llreadvllwritevopathfilehandlefile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 _ll_create created = 1 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 8 client.4555 _ll_create 0x1.head test_llreadvllwritevopathfilehandlefile44790 0666 66 = 0 (0x560b83f2e2c0 10000000002) 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 _ll_get 0x7f9dcc00adf0 0x10000000002 -> 1 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 fill_statx on 0x10000000002 snap/devhead mode 0100666 mtime 2024-03-28T01:08:16.560070+0000 ctime 2024-03-28T01:08:16.560070+0000 change_attr 0 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 3 client.4555 ll_release (fh)0x560b83f2e2c0 0x10000000002 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 8 client.4555 _release_fh 0x560b83f2e2c0 mode 3 on 0x10000000002.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 _flush 0x10000000002.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 nothing to flush 2024-03-28T01:08:16.572 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 check_caps on 0x10000000002.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) wanted - used - issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 delaying cap release 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 cap_delay_requeue on 0x10000000002.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 _release_fh 0x560b83f2e2c0 on inode 0x10000000002.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) no async_err state 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 3 client.4555 ll_unlink 0x1.head test_llreadvllwritevopathfilehandlefile44790 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 may_delete 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810); ; name test_llreadvllwritevopathfilehandlefile44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 _getattr mask As issued=1 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 3 client.4555 may_delete 0x7f9dcc00a810 = 0 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 8 client.4555 _unlink(0x1 test_llreadvllwritevopathfilehandlefile44790 uid 0 gid 0) 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 get_or_create 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) name test_llreadvllwritevopathfilehandlefile44790 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 _lookup have 0x1.head["test_llreadvllwritevopathfilehandlefile44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:16.573 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 _lookup 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) test_llreadvllwritevopathfilehandlefile44790 = 0x10000000002.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000002.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000002.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 choose_target_mds inode dir hash is 2 on test_llreadvllwritevopathfilehandlefile44790 => 3076465480 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 choose_target_mds 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810) is_hash=1 hash=3076465480 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 choose_target_mds from dirfragtree hash 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 mds is 0 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 encode_cap_releases enter (req: 0x560b8404f830, mds: 0) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 encode_inode_release enter(in:0x10000000002.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0), req:0x560b8404f830 mds:0, drop:LsxXx, unless:-, force:0) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 encode_dentry_release enter(dn:0x7f9de400d920) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00a810), req:0x560b8404f830 mds:0, drop:Fs, unless:Fx, force:1) 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 send_request set sent_stamp to 2024-03-28T01:08:16.565910+0000 2024-03-28T01:08:16.574 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 10 client.4555 send_request client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevopathfilehandlefile44790 2024-03-28T01:08:16.565871+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevopathfilehandlefile44790 2024-03-28T01:08:16.565871+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8404fb60 con 0x560b8405bb20 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9e0324a480 20 client.4555 awaiting reply|forward|kick on 0x7ffe68269860 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_caps(revoke ino 0x10000000002 1 seq 2 caps=pAsXsFsxcrwb dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:16.560070+0000 ctime 2024-03-28T01:08:16.560070+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de801d040 con 0x560b8405bb20 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 5 client.4555 set_cap_epoch_barrier epoch = 20 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 10 client.4555 mds.0 seq now 1 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 5 client.4555 handle_cap_grant on in 0x10000000002 mds.0 seq 2 caps now pAsXsFsxcrwb was pAsxXsFsxcrwb 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 10 client.4555 update_inode_file_time 0x10000000002.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) pAsxXsFsxcrwb ctime 2024-03-28T01:08:16.560070+0000 mtime 2024-03-28T01:08:16.560070+0000 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 10 client.4555 revocation of Ax 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 10 client.4555 check_caps on 0x10000000002.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) wanted - used - issued pAsXsFsxcrwb revoking Ax flags=1 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 10 client.4555 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsFsxcrwb revoking Ax 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 10 client.4555 completed revocation of Ax 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 10 client.4555 send_cap 0x10000000002.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevopathfilehandlefile44790"] 0x7f9dcc00adf0) mds.0 seq 2 used - want - flush - retain pAsLsXsFs held pAsxXsFsxcrwb revoking Ax dropping Fxcrwb 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 15 client.4555 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:16.575 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.565+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000002 1 seq 2 caps=pAsXsFs dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:16.560070+0000 ctime 2024-03-28T01:08:16.560070+0000 change_attr 0) v12 -- 0x7f9dcc00b3d0 con 0x560b8405bb20 2024-03-28T01:08:16.576 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.566+0000 7f9dfca00640 10 client.4555 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_reply(???:3 = 0 (0) Success unsafe) v1 ==== 513+0+0 (crc 0 0 0) 0x7f9ddc002810 con 0x560b8405bb20 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 handle_client_reply got a reply. Safe:0 tid 3 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 insert_trace from 2024-03-28T01:08:16.565910+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 features 0xffffffffffffffff 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 update_snap_trace len 70 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref was 2 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 get_snap_realm 0x1 0x7f9dcc00a720, nref now is 3 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 hrm is_target=0 is_dentry=1 2024-03-28T01:08:16.579 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 12 client.4555 add_update_inode had 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) caps pAsLsXs 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 update_inode_file_time 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=5 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) pAsLsXs ctime 2024-03-28T01:08:16.565871+0000 mtime 2024-03-28T01:08:16.565871+0000 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 dir hash is 2 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 12 client.4555 add_update_inode client inode change_attr: 5 , mds inodestat change_attr: 6 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 15 client.4555 unlink dir 0x7f9dcc00a810 'test_llreadvllwritevopathfilehandlefile44790' dn 0x7f9de400d920 inode 0x7f9dcc00adf0 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 unlink inode 0x7f9dcc00adf0 parents now 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 put_snap_realm 0x1 0x7f9dcc00a720 3 -> 2 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 handle_client_reply signalling caller 0x7ffe68269860 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 20 client.4555 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:16.580 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 20 client.4555 lat 0.002057 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 20 client.4555 trim_cache size 1 max 16384 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 8 client.4555 unlink(#0x1/test_llreadvllwritevopathfilehandlefile44790) = 0 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 2 client.4555 unmount 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 2 client.4555 unmounting 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83ef9790 con 0x560b8405bb20 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 10 client.4555 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size was 1+2 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 20 client.4555 trim_cache size 1 max 16384 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size still 1+2 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.567+0000 7f9e0324a480 10 client.4555 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_caps(grant ino 0x10000000002 1 seq 3 caps=pAsLsXsFsxcrwb dirty=- wanted=Ls follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:16.560070+0000 ctime 2024-03-28T01:08:16.565871+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dcc00b3d0 con 0x560b8405bb20 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 mds.0 seq now 2 2024-03-28T01:08:16.581 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 5 client.4555 handle_cap_grant on in 0x10000000002 mds.0 seq 3 caps now pAsLsXsFsxcrwb was pAsXsFs 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 update_inode_file_time 0x10000000002.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.560070+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) pAsXsFs ctime 2024-03-28T01:08:16.565871+0000 mtime 2024-03-28T01:08:16.560070+0000 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 max_size 4194304 -> 0 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 grant, new caps are LsFxcrwb 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 5 client.4555 _try_to_trim_inode in 0x10000000002.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size was 1+2 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 20 client.4555 trim_cache size 1 max 16384 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size still 1+2 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de802bdb0 con 0x560b8405bb20 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 20 client.4555 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 insert_trace from 2024-03-28T01:08:16.565910+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:16.582 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size was 1+2 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 20 client.4555 trim_cache size 1 max 16384 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 15 client.4555 trim_dentry unlinking dn test_llreadvllwritevopathfilehandlefile44790 in dir 0x1 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 15 client.4555 unlink dir 0x7f9dcc00a810 'test_llreadvllwritevopathfilehandlefile44790' dn 0x7f9de400d920 inode 0 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 15 client.4555 unlink removing 'test_llreadvllwritevopathfilehandlefile44790' dn 0x7f9de400d920 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 15 client.4555 close_dir dir 0x560b83f2ddc0 on 0x7f9dcc00a810 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_caps(grant ino 0x10000000002 1 seq 4 caps=pAsLsXsFsxcrwb dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:16.560070+0000 ctime 2024-03-28T01:08:16.565871+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de800a040 con 0x560b8405bb20 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _ll_drop_pins 2024-03-28T01:08:16.583 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 _ll_put 0x7f9dcc00adf0 0x10000000002 1 -> 0 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 _ll_put 0x7f9dcc00a810 0x1 2 -> 0 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 _release 0x10000000002.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _invalidate_inode_cache 0x10000000002.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _flush 0x10000000002.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 nothing to flush 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 _release 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.584 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _flush 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 nothing to flush 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x10000000002.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 1 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 1 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 flush_caps_sync 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 trim_cache size 0 max 16384 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) n = 11 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 15 client.4555 remove_cap last one, closing snaprealm 0x7f9dcc00a720 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_snap_realm 0x1 0x7f9dcc00a720 2 -> 1 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=- has_dir_layout 0x7f9dcc00a810) 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _put_inode on 0x10000000002.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) n = 15 2024-03-28T01:08:16.585 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 remove_cap mds.0 on 0x10000000002.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 15 client.4555 remove_cap last one, closing snaprealm 0x7f9dcc00a720 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 20 client.4555 put_snap_realm 0x1 0x7f9dcc00a720 1 -> 0 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 10 client.4555 _put_inode deleting 0x10000000002.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.560070+0000 mtime=2024-03-28T01:08:16.560070+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=1 caps=- objectset[0x10000000002 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00adf0) 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 2 client.4555 _close_mds_session mds.0 seq 2 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.572+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 2) v7 -- 0x560b83faadc0 con 0x560b8405bb20 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9e0324a480 2 client.4555 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9dfca00640 10 client.4555 mds.0 seq now 3 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 3) v7 -- 0x7f9dcc00a500 con 0x560b8405bb20 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9dfca00640 5 client.4555 handle_caps don't have vino 0x10000000002.head op 0, immediately releasing 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_cap_release(3) v2 -- 0x560b8404ff00 con 0x560b8405bb20 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size was 0+0 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9dfca00640 20 client.4555 trim_cache size 0 max 16384 2024-03-28T01:08:16.586 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.573+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size still 0+0 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc00a500 con 0x560b8405bb20 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 10 client.4555 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 5 client.4555 _closed_mds_session mds.0 seq 3 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 1 -- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 msgr2=0x560b83f2ca00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2ca00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 10 client.4555 remove_session_caps mds.0 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 10 client.4555 kick_requests_closed for mds.0 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size was 0+0 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 20 client.4555 trim_cache size 0 max 16384 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9dfca00640 10 client.4555 unmounting: trim pass, size still 0+0 2024-03-28T01:08:16.588 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 2 client.4555 unmounted. 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 client.4555 shutdown 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83ee5730 msgr2=0x560b8405a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83ee5730 0x560b8405a090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005e30 tx=0x7f9df0079000 comp rx=0 tx=0).stop 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 shutdown_connections 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2ca00 unknown :-1 s=CLOSED pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3401211103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83ee5730 0x560b8405a090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.577+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 >> 172.21.15.33:0/3401211103 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.578+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 shutdown_connections 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.578+0000 7f9e0324a480 1 -- 172.21.15.33:0/3401211103 wait complete. 2024-03-28T01:08:16.589 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.578+0000 7f9e0324a480 20 client.4555 trim_cache size 0 max 16384 2024-03-28T01:08:16.590 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritevOPathFileHandle (1043 ms) 2024-03-28T01:08:16.590 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritevReadOnlyFile 2024-03-28T01:08:16.593 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.579+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:16.593 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.580+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.580+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.580+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b84054150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.580+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f283a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.580+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc009cf0 con 0x560b83eff1f0 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.580+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8405a420 con 0x560b83f283a0 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.580+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbb070 con 0x560b83ee5730 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b84054150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b84054150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49450/0 (socket says 172.21.15.33:49450) 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 -- 172.21.15.33:0/99647728 learned_addr learned my addr 172.21.15.33:0/99647728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dffc00640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9e00600640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f283a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.594 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 -- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 -- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f283a0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f283a0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.581+0000 7f9dff200640 1 -- 172.21.15.33:0/99647728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f2ae90 con 0x560b83eff1f0 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9dff200640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b84054150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9de8001b80 tx=0x7f9de8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8388f9319d1eaf46 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8042660 con 0x560b83eff1f0 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de8042800 con 0x560b83eff1f0 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9e0324a480 10 client.4653 Subscribing to map 'mdsmap' 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83f28950 con 0x560b83eff1f0 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9df5600640 20 client.4653 tick 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9df5600640 20 client.4653 collect_and_send_metrics 2024-03-28T01:08:16.595 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9df5600640 20 client.4653 collect_and_send_global_metrics 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9df5600640 5 client.4653 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9df5600640 20 client.4653 trim_cache size 0 max 16384 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9df5600640 20 client.4653 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.582+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b8404ff00 con 0x560b83eff1f0 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025040 con 0x560b83eff1f0 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9dfde00640 1 client.4653 handle_mds_map epoch 8 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9e0324a480 20 client.4653 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9e0324a480 10 client.4653 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9e0324a480 20 client.4653 mds is 0 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9e0324a480 10 client.4653 _open_mds_session mds.0 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9e0324a480 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f2b9b0 con 0x560b8405bb20 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9e0324a480 10 client.4653 waiting for session to mds.0 to open 2024-03-28T01:08:16.596 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.583+0000 7f9dffc00640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.584+0000 7f9dffc00640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fb3f10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4759ab3602896f65 server_cookie=f210c4098d2d51ae in_seq=0 out_seq=0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.584+0000 7f9dfde00640 10 client.4653 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f2b9b0 con 0x560b8405bb20 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9dfde00640 10 client.4653 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9dfde00640 10 client.4653 renew_caps mds.0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc00a8b0 con 0x560b8405bb20 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9dfde00640 10 client.4653 connect_mds_targets for mds.0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 10 client.4653 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 20 client.4653 mds is 0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 10 client.4653 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 20 client.4653 encode_cap_releases enter (req: 0x560b83f2b1e0, mds: 0) 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 20 client.4653 send_request set sent_stamp to 2024-03-28T01:08:16.589246+0000 2024-03-28T01:08:16.597 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 10 client.4653 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.583820+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.583820+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb4530 con 0x560b8405bb20 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9e0324a480 20 client.4653 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc00a8b0 con 0x560b8405bb20 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.588+0000 7f9dfde00640 10 client.4653 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9dc4001630 con 0x560b8405bb20 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 insert_trace from 2024-03-28T01:08:16.589246+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 features 0xffffffffffffffff 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 update_snap_trace len 70 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 get_snap_realm 0x1 0x7f9dcc00aad0, nref now is 1 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.598 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 15 client.4653 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 15 client.4653 snapc 1=[] 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 put_snap_realm 0x1 0x7f9dcc00aad0 2 -> 1 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 hrm is_target=1 is_dentry=0 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00abc0) - ctime 2024-03-28T01:08:16.565871+0000 mtime 2024-03-28T01:08:16.565871+0000 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 dir hash is 2 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 12 client.4653 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 6 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 12 client.4653 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=- has_dir_layout 0x7f9dcc00abc0) caps pAsLsXsFs 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 get_snap_realm 0x1 0x7f9dcc00aad0, nref was 1 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 get_snap_realm 0x1 0x7f9dcc00aad0, nref now is 2 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 15 client.4653 add_update_cap first one, opened snaprealm 0x7f9dcc00aad0 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abc0) 2024-03-28T01:08:16.599 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abc0) 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 put_snap_realm 0x1 0x7f9dcc00aad0 2 -> 1 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 lat 0.000788 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 _ll_get 0x7f9dcc00abc0 0x1 -> 1 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 2 client.4653 unmount 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 unmounting: trim pass, size was 0+1 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 20 client.4653 trim_cache size 0 max 16384 2024-03-28T01:08:16.600 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9dfde00640 10 client.4653 unmounting: trim pass, size still 0+1 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 2 client.4653 unmounting 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 _ll_drop_pins 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 _ll_put 0x7f9dcc00abc0 0x1 1 -> 0 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 nothing to flush 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 1 2024-03-28T01:08:16.601 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 flush_caps_sync 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 trim_cache size 0 max 16384 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) n = 8 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abc0) 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 15 client.4653 remove_cap last one, closing snaprealm 0x7f9dcc00aad0 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 20 client.4653 put_snap_realm 0x1 0x7f9dcc00aad0 1 -> 0 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 10 client.4653 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=- COMPLETE has_dir_layout 0x7f9dcc00abc0) 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 2 client.4653 _close_mds_session mds.0 seq 0 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.602 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.589+0000 7f9e0324a480 2 client.4653 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9dfde00640 10 client.4653 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9dfde00640 5 client.4653 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9dfde00640 1 -- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 msgr2=0x560b83fb3f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9dfde00640 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fb3f10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9dfde00640 10 client.4653 remove_session_caps mds.0 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9dfde00640 10 client.4653 kick_requests_closed for mds.0 2024-03-28T01:08:16.607 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9e0324a480 2 client.4653 unmounted. 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.592+0000 7f9e0324a480 1 client.4653 shutdown 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 msgr2=0x560b84054150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b84054150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9de8001b80 tx=0x7f9de8001a30 comp rx=0 tx=0).stop 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 shutdown_connections 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f283a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83eff1f0 0x560b84054150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 --2- 172.21.15.33:0/99647728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 >> 172.21.15.33:0/99647728 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 shutdown_connections 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 1 -- 172.21.15.33:0/99647728 wait complete. 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.593+0000 7f9e0324a480 20 client.4653 trim_cache size 0 max 16384 2024-03-28T01:08:16.608 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.594+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.594+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83eff1f0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b84054190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc004730 con 0x560b83f283a0 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83fae1c0 con 0x560b83eff1f0 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2b100 con 0x560b83ee5730 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b84054190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b84054190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49464/0 (socket says 172.21.15.33:49464) 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9e00600640 1 -- 172.21.15.33:0/2384205907 learned_addr learned my addr 172.21.15.33:0/2384205907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.595+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9dff200640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83eff1f0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e00600640 1 -- 172.21.15.33:0/2384205907 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.609 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e00600640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e00600640 1 -- 172.21.15.33:0/2384205907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83eff1f0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e00600640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83eff1f0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e00600640 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83fad8c0 con 0x560b83f283a0 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e00600640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b84054190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001d30 tx=0x7f9df8024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3588971bdecfbc4e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8041080 con 0x560b83f283a0 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df8052bb0 con 0x560b83f283a0 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e0324a480 10 client.4656 Subscribing to map 'mdsmap' 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.596+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b8406bdf0 con 0x560b83f283a0 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9df6000640 20 client.4656 tick 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9df6000640 20 client.4656 collect_and_send_metrics 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9df6000640 20 client.4656 collect_and_send_global_metrics 2024-03-28T01:08:16.610 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9df6000640 5 client.4656 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9df6000640 20 client.4656 trim_cache size 0 max 16384 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9df6000640 20 client.4656 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df803f070 con 0x560b83f283a0 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b83f283a0 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9dfca00640 1 client.4656 handle_mds_map epoch 8 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9e0324a480 20 client.4656 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9e0324a480 10 client.4656 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9e0324a480 20 client.4656 mds is 0 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9e0324a480 10 client.4656 _open_mds_session mds.0 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9e0324a480 10 client.4656 waiting for session to mds.0 to open 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.597+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.611 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.598+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fad220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fd6567bc26f307 server_cookie=b1913297a0dee99f in_seq=0 out_seq=0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.598+0000 7f9dfca00640 10 client.4656 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9dfca00640 10 client.4656 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9dfca00640 10 client.4656 renew_caps mds.0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc00a8e0 con 0x560b8405bb20 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9dfca00640 10 client.4656 connect_mds_targets for mds.0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 10 client.4656 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 20 client.4656 mds is 0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 10 client.4656 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 20 client.4656 encode_cap_releases enter (req: 0x560b83ee4530, mds: 0) 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 20 client.4656 send_request set sent_stamp to 2024-03-28T01:08:16.602624+0000 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 10 client.4656 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.598021+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.598021+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa9220 con 0x560b8405bb20 2024-03-28T01:08:16.612 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.601+0000 7f9e0324a480 20 client.4656 awaiting reply|forward|kick on 0x7ffe68269840 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc00a8e0 con 0x560b8405bb20 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0005ad0 con 0x560b8405bb20 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 insert_trace from 2024-03-28T01:08:16.602624+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 features 0xffffffffffffffff 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 update_snap_trace len 70 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref now is 1 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 15 client.4656 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 15 client.4656 snapc 1=[] 2024-03-28T01:08:16.613 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 put_snap_realm 0x1 0x7f9dcc00ab00 2 -> 1 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 hrm is_target=1 is_dentry=0 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00abf0) - ctime 2024-03-28T01:08:16.565871+0000 mtime 2024-03-28T01:08:16.565871+0000 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 dir hash is 2 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 12 client.4656 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 6 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 12 client.4656 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=- has_dir_layout 0x7f9dcc00abf0) caps pAsLsXsFs 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref was 1 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref now is 2 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 15 client.4656 add_update_cap first one, opened snaprealm 0x7f9dcc00ab00 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 10 client.4656 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.614 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 put_snap_realm 0x1 0x7f9dcc00ab00 2 -> 1 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 handle_client_reply signalling caller 0x7ffe68269840 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 lat 0.000556 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 _ll_get 0x7f9dcc00abf0 0x1 -> 1 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 8 client.4656 _ll_create 0x1.head test_llreadvllwritevreadonlyfile44790 0666 576, uid 0, gid 0 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 10 client.4656 _lookup concluded ENOENT locally for 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0) dn 'test_llreadvllwritevreadonlyfile44790' 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 may_create 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 10 client.4656 _getattr mask As issued=1 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 3 client.4656 may_create 0x7f9dcc00abf0 = 0 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 8 client.4656 _create(0x1 test_llreadvllwritevreadonlyfile44790, 0666) 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 get_or_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0) name test_llreadvllwritevreadonlyfile44790 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 15 open_dir 0x560b83f2b7c0 on 0x7f9dcc00abf0 2024-03-28T01:08:16.615 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 15 client.4656 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0) 'test_llreadvllwritevreadonlyfile44790' dn 0x1.head["test_llreadvllwritevreadonlyfile44790"] (new dn) 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 choose_target_mds inode dir hash is 2 on test_llreadvllwritevreadonlyfile44790 => 1146310420 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 choose_target_mds 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0) is_hash=1 hash=1146310420 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 10 client.4656 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 mds is 0 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 10 client.4656 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 encode_cap_releases enter (req: 0x560b83fa9220, mds: 0) 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 encode_dentry_release enter(dn:0x7f9dcc00a720) 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00abf0), req:0x560b83fa9220 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 send_request set sent_stamp to 2024-03-28T01:08:16.603287+0000 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 10 client.4656 send_request client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevreadonlyfile44790 2024-03-28T01:08:16.603248+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevreadonlyfile44790 2024-03-28T01:08:16.603248+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa9710 con 0x560b8405bb20 2024-03-28T01:08:16.616 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9e0324a480 20 client.4656 awaiting reply|forward|kick on 0x7ffe68269680 2024-03-28T01:08:16.617 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.617 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.602+0000 7f9dfca00640 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.622 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 858+0+0 (crc 0 0 0) 0x7f9df0054bd0 con 0x560b8405bb20 2024-03-28T01:08:16.622 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 20 client.4656 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:16.622 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 insert_trace from 2024-03-28T01:08:16.603287+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:16.622 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 features 0xffffffffffffffff 2024-03-28T01:08:16.622 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 update_snap_trace len 70 2024-03-28T01:08:16.622 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref was 1 2024-03-28T01:08:16.622 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref now is 2 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 hrm is_target=1 is_dentry=1 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 update_inode_file_time 0x10000000003.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) - ctime 2024-03-28T01:08:16.603248+0000 mtime 2024-03-28T01:08:16.603248+0000 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 12 client.4656 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 12 client.4656 add_update_inode adding 0x10000000003.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=- objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) caps pAsxLsXsxFscr 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref was 2 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref now is 3 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 15 client.4656 add_update_cap first one, opened snaprealm 0x7f9dcc00ab00 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 add_update_cap issued - -> pAsxLsXsxFscr from mds.0 on 0x10000000003.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 12 client.4656 add_update_inode had 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00abf0) caps pAsLsXsFs 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.606+0000 7f9dfca00640 10 client.4656 update_inode_file_time 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.565871+0000 ctime=2024-03-28T01:08:16.565871+0000 change_attr=6 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00abf0) pAsLsXs ctime 2024-03-28T01:08:16.603248+0000 mtime 2024-03-28T01:08:16.603248+0000 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 dir hash is 2 2024-03-28T01:08:16.623 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 12 client.4656 add_update_inode client inode change_attr: 6 , mds inodestat change_attr: 7 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 10 client.4656 clearing (I_COMPLETE|I_DIR_ORDERED) on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 10 client.4656 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 10 client.4656 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 12 client.4656 insert_dentry_inode 'test_llreadvllwritevreadonlyfile44790' vino 0x10000000003.head in dir 0x1.head dn 0x7f9dcc00a720 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 15 client.4656 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) 'test_llreadvllwritevreadonlyfile44790' to inode 0x7f9dcc00b1d0 dn 0x1.head["test_llreadvllwritevreadonlyfile44790"] (old dn) 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 link inode 0x7f9dcc00b1d0 parents now 0x1.head["test_llreadvllwritevreadonlyfile44790"] 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 15 client.4656 update_dentry_lease 0x1.head["test_llreadvllwritevreadonlyfile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:16.603287+0000 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 put_snap_realm 0x1 0x7f9dcc00ab00 3 -> 2 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 handle_client_reply signalling caller 0x7ffe68269680 2024-03-28T01:08:16.624 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 10 client.4656 delegated_inos: [] 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 10 client.4656 make_request created ino 0x10000000003 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 make_request target is 0x10000000003.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 lat 0.004458 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000003.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000003.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 10 client.4656 _create_fh 0x10000000003 mode 1 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 trim_cache size 1 max 16384 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 8 client.4656 create(#0x1/test_llreadvllwritevreadonlyfile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 _ll_create created = 1 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 8 client.4656 _ll_create 0x1.head test_llreadvllwritevreadonlyfile44790 0666 576 = 0 (0x560b83fad910 10000000003) 2024-03-28T01:08:16.625 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 _ll_get 0x7f9dcc00b1d0 0x10000000003 -> 1 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 10 client.4656 fill_statx on 0x10000000003 snap/devhead mode 0100666 mtime 2024-03-28T01:08:16.603248+0000 ctime 2024-03-28T01:08:16.603248+0000 change_attr 0 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 3 client.4656 pwritev(0x560b83fad910, "...", 14, 0) = -9 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83faf310 0x560b83faf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- osd_op(unknown.0.0:1 3.32 3:4def72d4:::10000000003.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x560b8406c850 con 0x560b83faf310 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- osd_op(unknown.0.0:2 3.32 3:4def72d4:::10000000003.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x560b8406d0a0 con 0x560b83faf310 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dfca00640 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.607+0000 7f9dff200640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83faf310 0x560b83faf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.608+0000 7f9dff200640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83faf310 0x560b83faf790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=6ed15445a6a3a681 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.608+0000 7f9dfca00640 10 client.4656 ms_handle_connect on v2:172.21.15.33:6817/1178911672 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.608+0000 7f9dff200640 1 -- 172.21.15.33:0/2384205907 <== osd.0 v2:172.21.15.33:6817/1178911672 1 ==== osd_op_reply(1 10000000003.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de801e410 con 0x560b83faf310 2024-03-28T01:08:16.626 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dff200640 1 -- 172.21.15.33:0/2384205907 <== osd.0 v2:172.21.15.33:6817/1178911672 2 ==== osd_op_reply(2 10000000003.00000000 [create] v22'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de801e410 con 0x560b83faf310 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 get_caps 0x10000000003.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) have pAsxLsXsxFscr need Fr want Fc revoking - 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 _read_async 0x10000000003.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={2048=1} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 0~14 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 3 client.4656 preadv(0x560b83fad910, 0) = 0 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 3 client.4656 ll_release (fh)0x560b83fad910 0x10000000003 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 8 client.4656 _release_fh 0x560b83fad910 mode 1 on 0x10000000003.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={2048=0} open={1=1} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 _flush 0x10000000003.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 nothing to flush 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 check_caps on 0x10000000003.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) wanted - used - issued pAsxLsXsxFscr revoking - flags=0 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 cap mds.0 issued pAsxLsXsxFscr implemented pAsxLsXsxFscr revoking - 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 delaying cap release 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 cap_delay_requeue on 0x10000000003.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 _release_fh 0x560b83fad910 on inode 0x10000000003.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) no async_err state 2024-03-28T01:08:16.627 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 3 client.4656 ll_unlink 0x1.head test_llreadvllwritevreadonlyfile44790 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 may_delete 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0); ; name test_llreadvllwritevreadonlyfile44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 _getattr mask As issued=1 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 3 client.4656 may_delete 0x7f9dcc00abf0 = 0 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 8 client.4656 _unlink(0x1 test_llreadvllwritevreadonlyfile44790 uid 0 gid 0) 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 get_or_create 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) name test_llreadvllwritevreadonlyfile44790 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 _lookup have 0x1.head["test_llreadvllwritevreadonlyfile44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 _lookup 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) test_llreadvllwritevreadonlyfile44790 = 0x10000000003.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000003.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000003.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 choose_target_mds inode dir hash is 2 on test_llreadvllwritevreadonlyfile44790 => 1146310420 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 choose_target_mds 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0) is_hash=1 hash=1146310420 2024-03-28T01:08:16.628 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 choose_target_mds from dirfragtree hash 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 mds is 0 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 encode_cap_releases enter (req: 0x560b8406cbf0, mds: 0) 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 encode_inode_release enter(in:0x10000000003.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxLsXsxFscr(0=pAsxLsXsxFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0), req:0x560b8406cbf0 mds:0, drop:LsxXx, unless:-, force:0) 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 encode_dentry_release enter(dn:0x7f9dcc00a720) 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00abf0), req:0x560b8406cbf0 mds:0, drop:Fs, unless:Fx, force:1) 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 send_request set sent_stamp to 2024-03-28T01:08:16.610937+0000 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 10 client.4656 send_request client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevreadonlyfile44790 2024-03-28T01:08:16.610915+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 unlink #0x1/test_llreadvllwritevreadonlyfile44790 2024-03-28T01:08:16.610915+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa9b30 con 0x560b8405bb20 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9e0324a480 20 client.4656 awaiting reply|forward|kick on 0x7ffe68269860 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_caps(revoke ino 0x10000000003 1 seq 2 caps=pAsXsFscr dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:16.603248+0000 ctime 2024-03-28T01:08:16.603248+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081040 con 0x560b8405bb20 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 5 client.4656 set_cap_epoch_barrier epoch = 20 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 mds.0 seq now 1 2024-03-28T01:08:16.629 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 5 client.4656 handle_cap_grant on in 0x10000000003 mds.0 seq 2 caps now pAsXsFscr was pAsxXsFscr 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 update_inode_file_time 0x10000000003.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsxXsFscr(0=pAsxXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) pAsxXsFscr ctime 2024-03-28T01:08:16.603248+0000 mtime 2024-03-28T01:08:16.603248+0000 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 revocation of Ax 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 check_caps on 0x10000000003.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsXsFscr(0=pAsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) wanted - used - issued pAsXsFscr revoking Ax flags=1 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 cap mds.0 issued pAsXsFscr implemented pAsxXsFscr revoking Ax 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 completed revocation of Ax 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 send_cap 0x10000000003.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsXsFscr(0=pAsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevreadonlyfile44790"] 0x7f9dcc00b1d0) mds.0 seq 2 used - want - flush - retain pAsLsXsFscr held pAsxXsFscr revoking Ax dropping - 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 15 client.4656 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000003 1 seq 2 caps=pAsXsFscr dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:16.603248+0000 ctime 2024-03-28T01:08:16.603248+0000 change_attr 0) v12 -- 0x7f9dcc00b7b0 con 0x560b8405bb20 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.610+0000 7f9dfca00640 10 client.4656 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_reply(???:3 = 0 (0) Success unsafe) v1 ==== 506+0+0 (crc 0 0 0) 0x7f9df006c6e0 con 0x560b8405bb20 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 handle_client_reply got a reply. Safe:0 tid 3 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 insert_trace from 2024-03-28T01:08:16.610937+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:16.630 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 features 0xffffffffffffffff 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 update_snap_trace len 70 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref was 2 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 get_snap_realm 0x1 0x7f9dcc00ab00, nref now is 3 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 hrm is_target=0 is_dentry=1 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 12 client.4656 add_update_inode had 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) caps pAsLsXs 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 update_inode_file_time 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=7 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) pAsLsXs ctime 2024-03-28T01:08:16.610915+0000 mtime 2024-03-28T01:08:16.610915+0000 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 dir hash is 2 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 12 client.4656 add_update_inode client inode change_attr: 7 , mds inodestat change_attr: 8 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 15 client.4656 unlink dir 0x7f9dcc00abf0 'test_llreadvllwritevreadonlyfile44790' dn 0x7f9dcc00a720 inode 0x7f9dcc00b1d0 2024-03-28T01:08:16.631 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsXsFscr(0=pAsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 unlink inode 0x7f9dcc00b1d0 parents now 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsXsFscr(0=pAsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 put_snap_realm 0x1 0x7f9dcc00ab00 3 -> 2 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 handle_client_reply signalling caller 0x7ffe68269860 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 20 client.4656 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 20 client.4656 lat 0.001365 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 20 client.4656 trim_cache size 1 max 16384 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 8 client.4656 unlink(#0x1/test_llreadvllwritevreadonlyfile44790) = 0 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsXsFscr(0=pAsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 2 client.4656 unmount 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 2 client.4656 unmounting 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83f2b9b0 con 0x560b8405bb20 2024-03-28T01:08:16.632 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 10 client.4656 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, size was 1+2 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 20 client.4656 trim_cache size 1 max 16384 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, size still 1+2 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.611+0000 7f9e0324a480 10 client.4656 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_caps(grant ino 0x10000000003 1 seq 3 caps=pAsLsXsFscr dirty=- wanted=Ls follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:16.603248+0000 ctime 2024-03-28T01:08:16.610915+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dcc00b7b0 con 0x560b8405bb20 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 mds.0 seq now 2 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 5 client.4656 handle_cap_grant on in 0x10000000003 mds.0 seq 3 caps now pAsLsXsFscr was pAsXsFscr 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 update_inode_file_time 0x10000000003.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.603248+0000 change_attr=0 caps=pAsXsFscr(0=pAsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) pAsXsFscr ctime 2024-03-28T01:08:16.610915+0000 mtime 2024-03-28T01:08:16.603248+0000 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 grant, new caps are Ls 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 5 client.4656 _try_to_trim_inode in 0x10000000003.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, size was 1+2 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 20 client.4656 trim_cache size 1 max 16384 2024-03-28T01:08:16.633 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, size still 1+2 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df0054dc0 con 0x560b8405bb20 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 20 client.4656 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 insert_trace from 2024-03-28T01:08:16.610937+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, size was 1+2 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 20 client.4656 trim_cache size 1 max 16384 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 15 client.4656 trim_dentry unlinking dn test_llreadvllwritevreadonlyfile44790 in dir 0x1 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 15 client.4656 unlink dir 0x7f9dcc00abf0 'test_llreadvllwritevreadonlyfile44790' dn 0x7f9dcc00a720 inode 0 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 15 client.4656 unlink removing 'test_llreadvllwritevreadonlyfile44790' dn 0x7f9dcc00a720 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 15 client.4656 close_dir dir 0x560b83f2b7c0 on 0x7f9dcc00abf0 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.634 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _ll_drop_pins 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 _ll_put 0x7f9dcc00b1d0 0x10000000003 1 -> 0 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 _ll_put 0x7f9dcc00abf0 0x1 2 -> 0 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 _release 0x10000000003.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _invalidate_inode_cache 0x10000000003.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0,2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _flush 0x10000000003.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.635 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 nothing to flush 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 _release 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _flush 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 nothing to flush 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x10000000003.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 1 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 1 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 flush_caps_sync 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 trim_cache size 0 max 16384 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) n = 11 2024-03-28T01:08:16.636 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 15 client.4656 remove_cap last one, closing snaprealm 0x7f9dcc00ab00 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_snap_realm 0x1 0x7f9dcc00ab00 2 -> 1 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=- has_dir_layout 0x7f9dcc00abf0) 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _put_inode on 0x10000000003.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) n = 15 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 remove_cap mds.0 on 0x10000000003.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0,2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 15 client.4656 remove_cap last one, closing snaprealm 0x7f9dcc00ab00 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 20 client.4656 put_snap_realm 0x1 0x7f9dcc00ab00 1 -> 0 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 10 client.4656 _put_inode deleting 0x10000000003.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0,2048=0} open={1=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:16.603248+0000 mtime=2024-03-28T01:08:16.603248+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=1 caps=- objectset[0x10000000003 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b1d0) 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 2 client.4656 _close_mds_session mds.0 seq 2 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 2) v7 -- 0x560b83f2ac70 con 0x560b8405bb20 2024-03-28T01:08:16.637 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.617+0000 7f9e0324a480 2 client.4656 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83f2ac70 con 0x560b8405bb20 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 10 client.4656 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 5 client.4656 _closed_mds_session mds.0 seq 2 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 1 -- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 msgr2=0x560b83fad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83fad220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 10 client.4656 remove_session_caps mds.0 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 10 client.4656 kick_requests_closed for mds.0 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, size was 0+0 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 20 client.4656 trim_cache size 0 max 16384 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9dfca00640 10 client.4656 unmounting: trim pass, size still 0+0 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e0324a480 2 client.4656 unmounted. 2024-03-28T01:08:16.639 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e0324a480 1 client.4656 shutdown 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83faf310 msgr2=0x560b83faf790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83faf310 0x560b83faf790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 msgr2=0x560b84054190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2384205907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f283a0 0x560b84054190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001d30 tx=0x7f9df8024000 comp rx=0 tx=0).stop 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e00600640 1 -- 172.21.15.33:0/2384205907 reap_dead start 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.623+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 shutdown_connections 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.624+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 >> 172.21.15.33:0/2384205907 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.624+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 shutdown_connections 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.624+0000 7f9e0324a480 1 -- 172.21.15.33:0/2384205907 wait complete. 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.624+0000 7f9e0324a480 20 client.4656 trim_cache size 0 max 16384 2024-03-28T01:08:16.640 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritevReadOnlyFile (46 ms) 2024-03-28T01:08:16.641 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritevIOClientNotMounted 2024-03-28T01:08:16.648 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.624+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:16.648 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.625+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:16.648 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.625+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.648 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.625+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83f283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.625+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.625+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc004a70 con 0x560b8405bb20 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.625+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2e680 con 0x560b83fadf60 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.625+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83eff1f0 con 0x560b83ee5730 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49476/0 (socket says 172.21.15.33:49476) 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 -- 172.21.15.33:0/9458590 learned_addr learned my addr 172.21.15.33:0/9458590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9dffc00640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9dff200640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83f283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 -- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 msgr2=0x560b83f283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83f283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 -- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.649 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9e00600640 1 -- 172.21.15.33:0/9458590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83fb89b0 con 0x560b8405bb20 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.626+0000 7f9dff200640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83f283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e00600640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b84059ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000f80 tx=0x7f9df803d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cb7f9a8f20a2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8041070 con 0x560b8405bb20 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df8052bb0 con 0x560b8405bb20 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 10 client.4659 Subscribing to map 'mdsmap' 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b8404f6f0 con 0x560b8405bb20 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9df5600640 20 client.4659 tick 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9df5600640 20 client.4659 collect_and_send_metrics 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9df5600640 20 client.4659 collect_and_send_global_metrics 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9df5600640 5 client.4659 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9df5600640 20 client.4659 trim_cache size 0 max 16384 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9df5600640 20 client.4659 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:16.650 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8052dd0 con 0x560b8405bb20 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b8405bb20 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9dfde00640 1 client.4659 handle_mds_map epoch 8 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 20 client.4659 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 10 client.4659 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 20 client.4659 mds is 0 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 10 client.4659 _open_mds_session mds.0 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fb16a0 0x560b83f292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f2ac70 con 0x560b83fb16a0 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.627+0000 7f9e0324a480 10 client.4659 waiting for session to mds.0 to open 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.628+0000 7f9dffc00640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fb16a0 0x560b83f292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.628+0000 7f9dffc00640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fb16a0 0x560b83f292f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a986e5400850bf95 server_cookie=d568931cb385c8c7 in_seq=0 out_seq=0 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.628+0000 7f9dfde00640 10 client.4659 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:16.651 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f2ac70 con 0x560b83fb16a0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 renew_caps mds.0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc008240 con 0x560b83fb16a0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 connect_mds_targets for mds.0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 10 client.4659 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 20 client.4659 mds is 0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 10 client.4659 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 20 client.4659 encode_cap_releases enter (req: 0x560b84054190, mds: 0) 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 20 client.4659 send_request set sent_stamp to 2024-03-28T01:08:16.633206+0000 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 10 client.4659 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.628576+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.628576+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2b280 con 0x560b83fb16a0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9e0324a480 20 client.4659 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc008240 con 0x560b83fb16a0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:16.652 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b83fb16a0 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 20 client.4659 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 insert_trace from 2024-03-28T01:08:16.633206+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 features 0xffffffffffffffff 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 update_snap_trace len 70 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 20 client.4659 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 20 client.4659 get_snap_realm 0x1 0x7f9dcc00abf0, nref now is 1 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 15 client.4659 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 15 client.4659 snapc 1=[] 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 20 client.4659 put_snap_realm 0x1 0x7f9dcc00abf0 2 -> 1 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 hrm is_target=1 is_dentry=0 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 10 client.4659 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00ace0) - ctime 2024-03-28T01:08:16.610915+0000 mtime 2024-03-28T01:08:16.610915+0000 2024-03-28T01:08:16.653 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 20 client.4659 dir hash is 2 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 12 client.4659 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 8 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.632+0000 7f9dfde00640 12 client.4659 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=- has_dir_layout 0x7f9dcc00ace0) caps pAsLsXsFs 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 get_snap_realm 0x1 0x7f9dcc00abf0, nref was 1 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 get_snap_realm 0x1 0x7f9dcc00abf0, nref now is 2 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 15 client.4659 add_update_cap first one, opened snaprealm 0x7f9dcc00abf0 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 10 client.4659 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 10 client.4659 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 10 client.4659 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 put_snap_realm 0x1 0x7f9dcc00abf0 2 -> 1 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 lat 0.000567 2024-03-28T01:08:16.654 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 _ll_get 0x7f9dcc00ace0 0x1 -> 1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 2 client.4659 unmount 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 10 client.4659 unmounting: trim pass, size was 0+1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 20 client.4659 trim_cache size 0 max 16384 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9dfde00640 10 client.4659 unmounting: trim pass, size still 0+1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 2 client.4659 unmounting 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 _ll_drop_pins 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 _ll_put 0x7f9dcc00ace0 0x1 1 -> 0 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.655 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 nothing to flush 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 flush_caps_sync 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 trim_cache size 0 max 16384 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 8 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 15 client.4659 remove_cap last one, closing snaprealm 0x7f9dcc00abf0 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 20 client.4659 put_snap_realm 0x1 0x7f9dcc00abf0 1 -> 0 2024-03-28T01:08:16.656 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 10 client.4659 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=- COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 2 client.4659 _close_mds_session mds.0 seq 0 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83f27fb0 con 0x560b83fb16a0 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.633+0000 7f9e0324a480 2 client.4659 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83f27fb0 con 0x560b83fb16a0 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9dfde00640 10 client.4659 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9dfde00640 5 client.4659 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9dfde00640 1 -- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fb16a0 msgr2=0x560b83f292f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9dfde00640 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fb16a0 0x560b83f292f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9dfde00640 10 client.4659 remove_session_caps mds.0 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9dfde00640 10 client.4659 kick_requests_closed for mds.0 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 2 client.4659 unmounted. 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 client.4659 shutdown 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 msgr2=0x560b84059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:16.657 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b84059ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000f80 tx=0x7f9df803d070 comp rx=0 tx=0).stop 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 shutdown_connections 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fb16a0 0x560b83f292f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b84059ee0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83f283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 --2- 172.21.15.33:0/9458590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 >> 172.21.15.33:0/9458590 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 shutdown_connections 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 1 -- 172.21.15.33:0/9458590 wait complete. 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.637+0000 7f9e0324a480 20 client.4659 trim_cache size 0 max 16384 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.639+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.640+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.641+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.641+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.658 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.641+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.641+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc001270 con 0x560b83ee5730 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.641+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f31f90 con 0x560b83fadf60 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.641+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83faa550 con 0x560b83fb16a0 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:34038/0 (socket says 172.21.15.33:34038) 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 -- 172.21.15.33:0/261115838 learned_addr learned my addr 172.21.15.33:0/261115838 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9e00600640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dffc00640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 -- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 msgr2=0x560b83f283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 -- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.642+0000 7f9dff200640 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f2ce70 con 0x560b83fadf60 2024-03-28T01:08:16.659 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9e0324a480 10 client.4567 Subscribing to map 'mdsmap' 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mdsmap=0+}) v3 -- 0x560b8406cdc0 con 0x560b83fadf60 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9dff200640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b84059ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000fd0 tx=0x560b83f2ddc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8742e1e552efd17b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9dfca00640 10 client.4567 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9df6000640 20 client.4567 tick 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9df6000640 20 client.4567 collect_and_send_metrics 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9df6000640 20 client.4567 collect_and_send_global_metrics 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9df6000640 5 client.4567 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9df6000640 20 client.4567 trim_cache size 0 max 16384 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.643+0000 7f9df6000640 20 client.4567 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8065060 con 0x560b83fadf60 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de8042870 con 0x560b83fadf60 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025040 con 0x560b83fadf60 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9dfca00640 1 client.4567 handle_mds_map epoch 8 2024-03-28T01:08:16.660 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9e0324a480 20 client.4567 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9e0324a480 10 client.4567 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9e0324a480 20 client.4567 mds is 0 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9e0324a480 10 client.4567 _open_mds_session mds.0 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9e0324a480 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f29dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.644+0000 7f9e0324a480 10 client.4567 waiting for session to mds.0 to open 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.645+0000 7f9dffc00640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f29dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.645+0000 7f9dffc00640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f29dd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45559b5a593f43bb server_cookie=770e39d9b1b90962 in_seq=0 out_seq=0 2024-03-28T01:08:16.661 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.645+0000 7f9dfca00640 10 client.4567 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 renew_caps mds.0 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc0081c0 con 0x560b8405bb20 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 connect_mds_targets for mds.0 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 mds is 0 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 encode_cap_releases enter (req: 0x560b83fae540, mds: 0) 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 send_request set sent_stamp to 2024-03-28T01:08:16.650780+0000 2024-03-28T01:08:16.666 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.644964+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.644964+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2b100 con 0x560b8405bb20 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 awaiting reply|forward|kick on 0x7ffe68269840 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc0081c0 con 0x560b8405bb20 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b8405bb20 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 insert_trace from 2024-03-28T01:08:16.650780+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 features 0xffffffffffffffff 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 update_snap_trace len 70 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0x7f9dcc00abf0, nref now is 1 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.667 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 15 client.4567 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 15 client.4567 snapc 1=[] 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 put_snap_realm 0x1 0x7f9dcc00abf0 2 -> 1 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 hrm is_target=1 is_dentry=0 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00ace0) - ctime 2024-03-28T01:08:16.610915+0000 mtime 2024-03-28T01:08:16.610915+0000 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 dir hash is 2 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 12 client.4567 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 8 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 12 client.4567 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=- has_dir_layout 0x7f9dcc00ace0) caps pAsLsXsFs 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0x7f9dcc00abf0, nref was 1 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0x7f9dcc00abf0, nref now is 2 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 15 client.4567 add_update_cap first one, opened snaprealm 0x7f9dcc00abf0 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 10 client.4567 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 put_snap_realm 0x1 0x7f9dcc00abf0 2 -> 1 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 handle_client_reply signalling caller 0x7ffe68269840 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 lat 0.000641 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 _ll_get 0x7f9dcc00ace0 0x1 -> 1 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 8 client.4567 _ll_create 0x1.head test_llreadvllwritevioclientnotmountedfile44790 0666 578, uid 0, gid 0 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 _lookup concluded ENOENT locally for 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) dn 'test_llreadvllwritevioclientnotmountedfile44790' 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 may_create 0x1.head(faked_ino=0 nref=6 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 _getattr mask As issued=1 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 3 client.4567 may_create 0x7f9dcc00ace0 = 0 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 8 client.4567 _create(0x1 test_llreadvllwritevioclientnotmountedfile44790, 0666) 2024-03-28T01:08:16.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 get_or_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) name test_llreadvllwritevioclientnotmountedfile44790 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 15 open_dir 0x560b8406ce30 on 0x7f9dcc00ace0 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 15 client.4567 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 'test_llreadvllwritevioclientnotmountedfile44790' dn 0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] (new dn) 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 choose_target_mds inode dir hash is 2 on test_llreadvllwritevioclientnotmountedfile44790 => 3240440152 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 choose_target_mds 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) is_hash=1 hash=3240440152 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 mds is 0 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 encode_cap_releases enter (req: 0x560b83f2b100, mds: 0) 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 encode_dentry_release enter(dn:0x7f9dcc00ab00) 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE has_dir_layout 0x7f9dcc00ace0), req:0x560b83f2b100 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 send_request set sent_stamp to 2024-03-28T01:08:16.651516+0000 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 10 client.4567 send_request client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevioclientnotmountedfile44790 2024-03-28T01:08:16.651484+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevioclientnotmountedfile44790 2024-03-28T01:08:16.651484+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2b480 con 0x560b8405bb20 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9e0324a480 20 client.4567 awaiting reply|forward|kick on 0x7ffe68269680 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.650+0000 7f9dfca00640 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 868+0+0 (crc 0 0 0) 0x7f9df0002040 con 0x560b8405bb20 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 insert_trace from 2024-03-28T01:08:16.651516+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 features 0xffffffffffffffff 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 update_snap_trace len 70 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0x7f9dcc00abf0, nref was 1 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0x7f9dcc00abf0, nref now is 2 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 hrm is_target=1 is_dentry=1 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 update_inode_file_time 0x10000000004.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) - ctime 2024-03-28T01:08:16.651484+0000 mtime 2024-03-28T01:08:16.651484+0000 2024-03-28T01:08:16.671 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 12 client.4567 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 12 client.4567 add_update_inode adding 0x10000000004.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=- objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0x7f9dcc00abf0, nref was 2 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 get_snap_realm 0x1 0x7f9dcc00abf0, nref now is 3 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 15 client.4567 add_update_cap first one, opened snaprealm 0x7f9dcc00abf0 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000004.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 12 client.4567 add_update_inode had 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00ace0) caps pAsLsXsFs 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 update_inode_file_time 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.610915+0000 ctime=2024-03-28T01:08:16.610915+0000 change_attr=8 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00ace0) pAsLsXs ctime 2024-03-28T01:08:16.651484+0000 mtime 2024-03-28T01:08:16.651484+0000 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 dir hash is 2 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 12 client.4567 add_update_inode client inode change_attr: 8 , mds inodestat change_attr: 9 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 clearing (I_COMPLETE|I_DIR_ORDERED) on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXs(0=pAsLsXs) COMPLETE has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.672 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 12 client.4567 insert_dentry_inode 'test_llreadvllwritevioclientnotmountedfile44790' vino 0x10000000004.head in dir 0x1.head dn 0x7f9dcc00ab00 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 15 client.4567 link dir 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 'test_llreadvllwritevioclientnotmountedfile44790' to inode 0x7f9dcc00b2c0 dn 0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] (old dn) 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 link inode 0x7f9dcc00b2c0 parents now 0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 15 client.4567 update_dentry_lease 0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:16.651516+0000 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_snap_realm 0x1 0x7f9dcc00abf0 3 -> 2 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 handle_client_reply signalling caller 0x7ffe68269680 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 delegated_inos: [] 2024-03-28T01:08:16.673 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 make_request created ino 0x10000000004 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 make_request target is 0x10000000004.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 lat 0.004608 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000004.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000004.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _create_fh 0x10000000004 mode 3 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 trim_cache size 1 max 16384 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 8 client.4567 create(#0x1/test_llreadvllwritevioclientnotmountedfile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 _ll_create created = 1 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 8 client.4567 _ll_create 0x1.head test_llreadvllwritevioclientnotmountedfile44790 0666 578 = 0 (0x560b83f2b480 10000000004) 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 _ll_get 0x7f9dcc00b2c0 0x10000000004 -> 1 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 fill_statx on 0x10000000004 snap/devhead mode 0100666 mtime 2024-03-28T01:08:16.651484+0000 ctime 2024-03-28T01:08:16.651484+0000 change_attr 0 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.674 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 3 client.4567 ll_release (fh)0x560b83f2b480 0x10000000004 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 8 client.4567 _release_fh 0x560b83f2b480 mode 3 on 0x10000000004.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _flush 0x10000000004.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 nothing to flush 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 check_caps on 0x10000000004.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) wanted - used - issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 delaying cap release 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 cap_delay_requeue on 0x10000000004.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _release_fh 0x560b83f2b480 on inode 0x10000000004.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) no async_err state 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 2 client.4567 unmount 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 2 client.4567 unmounting 2024-03-28T01:08:16.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83f2ac70 con 0x560b8405bb20 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 waiting for write request '2' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevioclientnotmountedfile44790"] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 unmounting: trim pass, size was 1+2 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 trim_cache size 1 max 16384 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 15 client.4567 trim_dentry unlinking dn test_llreadvllwritevioclientnotmountedfile44790 in dir 0x1 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 15 client.4567 unlink dir 0x7f9dcc00ace0 'test_llreadvllwritevioclientnotmountedfile44790' dn 0x7f9dcc00ab00 inode 0x7f9dcc00b2c0 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 unlink inode 0x7f9dcc00b2c0 parents now 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 15 client.4567 unlink removing 'test_llreadvllwritevioclientnotmountedfile44790' dn 0x7f9dcc00ab00 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 15 client.4567 close_dir dir 0x560b8406ce30 on 0x7f9dcc00ace0 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9dfca00640 10 client.4567 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _ll_drop_pins 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 _ll_put 0x7f9dcc00b2c0 0x10000000004 1 -> 0 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 _ll_put 0x7f9dcc00ace0 0x1 2 -> 0 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 _release 0x10000000004.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _invalidate_inode_cache 0x10000000004.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _flush 0x10000000004.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 nothing to flush 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 _release 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _flush 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 nothing to flush 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x10000000004.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) n = 1 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 1 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 flush_caps_sync 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 trim_cache size 0 max 16384 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) n = 10 2024-03-28T01:08:16.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 15 client.4567 remove_cap last one, closing snaprealm 0x7f9dcc00abf0 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_snap_realm 0x1 0x7f9dcc00abf0 2 -> 1 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=- has_dir_layout 0x7f9dcc00ace0) 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _put_inode on 0x10000000004.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) n = 13 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 remove_cap mds.0 on 0x10000000004.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 15 client.4567 remove_cap last one, closing snaprealm 0x7f9dcc00abf0 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 20 client.4567 put_snap_realm 0x1 0x7f9dcc00abf0 1 -> 0 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 10 client.4567 _put_inode deleting 0x10000000004.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:16.651484+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=0 caps=- objectset[0x10000000004 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b2c0) 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 2 client.4567 _close_mds_session mds.0 seq 0 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83fbeee0 con 0x560b8405bb20 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.655+0000 7f9e0324a480 2 client.4567 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83fbeee0 con 0x560b8405bb20 2024-03-28T01:08:16.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9dfca00640 10 client.4567 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9dfca00640 5 client.4567 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9dfca00640 1 -- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 msgr2=0x560b83f29dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9dfca00640 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f29dd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9dfca00640 10 client.4567 remove_session_caps mds.0 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9dfca00640 10 client.4567 kick_requests_closed for mds.0 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 2 client.4567 unmounted. 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 client.4567 shutdown 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 msgr2=0x560b84059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b84059ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000fd0 tx=0x560b83f2ddc0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 shutdown_connections 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f29dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 --2- 172.21.15.33:0/261115838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.681 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 >> 172.21.15.33:0/261115838 conn(0x560b83f2d970 msgr2=0x560b83f29260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:16.681 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 shutdown_connections 2024-03-28T01:08:16.681 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 1 -- 172.21.15.33:0/261115838 wait complete. 2024-03-28T01:08:16.681 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.661+0000 7f9e0324a480 20 client.4567 trim_cache size 0 max 16384 2024-03-28T01:08:16.681 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritevIOClientNotMounted (37 ms) 2024-03-28T01:08:16.681 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritevNegativeIOVCount 2024-03-28T01:08:16.685 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.662+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:16.685 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.663+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.663+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.663+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.663+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc001350 con 0x560b83ee5730 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f277f0 con 0x560b83fb16a0 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83faa550 con 0x560b83fadf60 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49498/0 (socket says 172.21.15.33:49498) 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 -- 172.21.15.33:0/423230789 learned_addr learned my addr 172.21.15.33:0/423230789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dff200640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9e00600640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 -- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fadf60 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.686 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 -- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.664+0000 7f9dffc00640 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f2c870 con 0x560b83ee5730 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9dffc00640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9df000e210 tx=0x560b84055580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc3f634e1a5be49 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0099080 con 0x560b83ee5730 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 10 client.4665 Subscribing to map 'mdsmap' 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df0005ad0 con 0x560b83ee5730 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83eff3c0 con 0x560b83ee5730 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9df5600640 20 client.4665 tick 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9df5600640 20 client.4665 collect_and_send_metrics 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9df5600640 20 client.4665 collect_and_send_global_metrics 2024-03-28T01:08:16.687 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9df5600640 5 client.4665 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9df5600640 20 client.4665 trim_cache size 0 max 16384 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9df5600640 20 client.4665 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0017dd0 con 0x560b83ee5730 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0091040 con 0x560b83ee5730 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9dfde00640 1 client.4665 handle_mds_map epoch 8 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 20 client.4665 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 10 client.4665 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 20 client.4665 mds is 0 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 10 client.4665 _open_mds_session mds.0 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b8405a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.665+0000 7f9e0324a480 10 client.4665 waiting for session to mds.0 to open 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.666+0000 7f9dff200640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b8405a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.666+0000 7f9dff200640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b8405a700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae175c2c47254c9 server_cookie=208434690d098601 in_seq=0 out_seq=0 2024-03-28T01:08:16.688 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.666+0000 7f9dfde00640 10 client.4665 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83f27fb0 con 0x560b8405bb20 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9dfde00640 10 client.4665 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9dfde00640 10 client.4665 renew_caps mds.0 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc0090a0 con 0x560b8405bb20 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9dfde00640 10 client.4665 connect_mds_targets for mds.0 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 10 client.4665 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 20 client.4665 mds is 0 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 10 client.4665 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 20 client.4665 encode_cap_releases enter (req: 0x560b84054210, mds: 0) 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 20 client.4665 send_request set sent_stamp to 2024-03-28T01:08:16.671515+0000 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 10 client.4665 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.666542+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:16.666542+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa95e0 con 0x560b8405bb20 2024-03-28T01:08:16.689 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.670+0000 7f9e0324a480 20 client.4665 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b8405bb20 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de8000c30 con 0x560b8405bb20 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 insert_trace from 2024-03-28T01:08:16.671515+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 features 0xffffffffffffffff 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 update_snap_trace len 70 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 get_snap_realm 0x1 0x7f9dcc00ace0, nref now is 1 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 15 client.4665 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 15 client.4665 snapc 1=[] 2024-03-28T01:08:16.690 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 put_snap_realm 0x1 0x7f9dcc00ace0 2 -> 1 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 hrm is_target=1 is_dentry=0 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00add0) - ctime 2024-03-28T01:08:16.651484+0000 mtime 2024-03-28T01:08:16.651484+0000 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 dir hash is 2 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 12 client.4665 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 9 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 12 client.4665 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=- has_dir_layout 0x7f9dcc00add0) caps pAsLsXsFs 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 get_snap_realm 0x1 0x7f9dcc00ace0, nref was 1 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 get_snap_realm 0x1 0x7f9dcc00ace0, nref now is 2 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 15 client.4665 add_update_cap first one, opened snaprealm 0x7f9dcc00ace0 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 add_update_cap calling signal_caps_inode 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 put_snap_realm 0x1 0x7f9dcc00ace0 2 -> 1 2024-03-28T01:08:16.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 lat 0.000585 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 _ll_get 0x7f9dcc00add0 0x1 -> 1 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 2 client.4665 unmount 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 2 client.4665 unmounting 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b84054580 con 0x560b8405bb20 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 10 client.4665 _ll_drop_pins 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 _ll_put 0x7f9dcc00add0 0x1 1 -> 0 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 10 client.4665 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 10 client.4665 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 10 client.4665 nothing to flush 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 10 client.4665 flush_caps_sync 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 10 client.4665 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 20 client.4665 trim_cache size 0 max 16384 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 10 client.4665 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 6 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 2 client.4665 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 1 2024-03-28T01:08:16.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 unmounting: trim pass, size was 0+1 2024-03-28T01:08:16.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 20 client.4665 trim_cache size 0 max 16384 2024-03-28T01:08:16.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9dfde00640 10 client.4665 unmounting: trim pass, size still 0+1 2024-03-28T01:08:16.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:16.671+0000 7f9e0324a480 2 client.4665 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:17.668 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 20 client.4665 tick 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 10 client.4665 renew_caps() 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 15 client.4665 renew_caps requesting from mds.0 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 10 client.4665 renew_caps mds.0 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 2) v7 -- 0x7f9dc4001910 con 0x560b8405bb20 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 20 client.4665 collect_and_send_metrics 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 20 client.4665 collect_and_send_global_metrics 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_metrics [client_metric_type: READ_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: WRITE_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: METADATA_LATENCY latency: 0.000585, avg_latency: 0.000585, sq_sum: 0, count=1][client_metric_type: CAP_INFO cap_hits: 0 cap_misses: 0 num_caps: 0][client_metric_type: DENTRY_LEASE dlease_hits: 0 dlease_misses: 0 num_dentries: 0][client_metric_type: OPENED_FILES opened_files: 0 total_inodes: 1][client_metric_type: PINNED_ICAPS pinned_icaps: 1 total_inodes: 1][client_metric_type: OPENED_INODES opened_inodes: 0 total_inodes: 1][client_metric_type: READ_IO_SIZES total_ops: 0 total_size: 0][client_metric_type: WRITE_IO_SIZES total_ops: 0 total_size: 0] v1 -- 0x7f9dc40012c0 con 0x560b8405bb20 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 10 client.4665 _put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) n = 2 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 10 client.4665 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00add0) 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 15 client.4665 remove_cap last one, closing snaprealm 0x7f9dcc00ace0 2024-03-28T01:08:17.669 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 20 client.4665 put_snap_realm 0x1 0x7f9dcc00ace0 1 -> 0 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 10 client.4665 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=- has_dir_layout 0x7f9dcc00add0) 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 20 client.4665 trim_cache size 0 max 16384 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9df5600640 20 client.4665 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9e0324a480 2 client.4665 _close_mds_session mds.0 seq 0 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83fa95e0 con 0x560b8405bb20 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.665+0000 7f9e0324a480 2 client.4665 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.666+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (crc 0 0 0) 0x560b84054580 con 0x560b8405bb20 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.666+0000 7f9dfde00640 10 client.4665 handle_client_session client_session(renewcaps seq 2) v1 from mds.0 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.666+0000 7f9dfde00640 10 client.4665 unmounting: trim pass, size was 0+0 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.666+0000 7f9dfde00640 20 client.4665 trim_cache size 0 max 16384 2024-03-28T01:08:17.670 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.666+0000 7f9dfde00640 10 client.4665 unmounting: trim pass, size still 0+0 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83fa95e0 con 0x560b8405bb20 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 10 client.4665 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 5 client.4665 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 1 -- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 msgr2=0x560b8405a700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b8405a700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 10 client.4665 remove_session_caps mds.0 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 10 client.4665 kick_requests_closed for mds.0 2024-03-28T01:08:17.675 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 10 client.4665 unmounting: trim pass, size was 0+0 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 20 client.4665 trim_cache size 0 max 16384 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9dfde00640 10 client.4665 unmounting: trim pass, size still 0+0 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9e0324a480 2 client.4665 unmounted. 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9e0324a480 1 client.4665 shutdown 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.672+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 msgr2=0x560b83fb3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9df000e210 tx=0x560b84055580 comp rx=0 tx=0).stop 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 shutdown_connections 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b8405a700 unknown :-1 s=CLOSED pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fadf60 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 --2- 172.21.15.33:0/423230789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.676 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 >> 172.21.15.33:0/423230789 conn(0x560b83f2d970 msgr2=0x560b83f29dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:17.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 shutdown_connections 2024-03-28T01:08:17.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 1 -- 172.21.15.33:0/423230789 wait complete. 2024-03-28T01:08:17.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.673+0000 7f9e0324a480 20 client.4665 trim_cache size 0 max 16384 2024-03-28T01:08:17.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.674+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:17.677 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.675+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:17.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.676+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.676+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.678 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.676+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.676+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc004a10 con 0x560b83fadf60 2024-03-28T01:08:17.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.676+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2dd80 con 0x560b83fb16a0 2024-03-28T01:08:17.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.676+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fae710 con 0x560b83ee5730 2024-03-28T01:08:17.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9e00600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.679 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:44094/0 (socket says 172.21.15.33:44094) 2024-03-28T01:08:17.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9e00600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:34056/0 (socket says 172.21.15.33:34056) 2024-03-28T01:08:17.680 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dffc00640 1 -- 172.21.15.33:0/4143779741 learned_addr learned my addr 172.21.15.33:0/4143779741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dff200640 1 -- 172.21.15.33:0/4143779741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dff200640 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dff200640 1 -- 172.21.15.33:0/4143779741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.677+0000 7f9dff200640 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9dff200640 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8406bdf0 con 0x560b83fadf60 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9e00600640 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9dff200640 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x560b8405a630 tx=0x7f9de8051040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948e3a07b8d1423d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40012c0 con 0x560b83fadf60 2024-03-28T01:08:17.682 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de802b8b0 con 0x560b83fadf60 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9e0324a480 10 client.4668 Subscribing to map 'mdsmap' 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.678+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83fae540 con 0x560b83fadf60 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9df6000640 20 client.4668 tick 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9df6000640 20 client.4668 collect_and_send_metrics 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9df6000640 20 client.4668 collect_and_send_global_metrics 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9df6000640 5 client.4668 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9df6000640 20 client.4668 trim_cache size 0 max 16384 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9df6000640 20 client.4668 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de803d070 con 0x560b83fadf60 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025040 con 0x560b83fadf60 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9dfca00640 1 client.4668 handle_mds_map epoch 8 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9e0324a480 20 client.4668 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9e0324a480 10 client.4668 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9e0324a480 20 client.4668 mds is 0 2024-03-28T01:08:17.683 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9e0324a480 10 client.4668 _open_mds_session mds.0 2024-03-28T01:08:17.684 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.684 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b8405bb20 2024-03-28T01:08:17.684 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.679+0000 7f9e0324a480 10 client.4668 waiting for session to mds.0 to open 2024-03-28T01:08:17.684 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.680+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.684 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.681+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2e280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d2e21e4edd83 server_cookie=339f761405851608 in_seq=0 out_seq=0 2024-03-28T01:08:17.684 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.681+0000 7f9dfca00640 10 client.4668 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b8405bb20 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9dfca00640 10 client.4668 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9dfca00640 10 client.4668 renew_caps mds.0 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc00ace0 con 0x560b8405bb20 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9dfca00640 10 client.4668 connect_mds_targets for mds.0 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 10 client.4668 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 20 client.4668 mds is 0 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 10 client.4668 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 20 client.4668 encode_cap_releases enter (req: 0x560b83fae540, mds: 0) 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 20 client.4668 send_request set sent_stamp to 2024-03-28T01:08:17.687254+0000 2024-03-28T01:08:17.691 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 10 client.4668 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:17.680275+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:17.680275+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b840544a0 con 0x560b8405bb20 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9e0324a480 20 client.4668 awaiting reply|forward|kick on 0x7ffe68269830 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc00ace0 con 0x560b8405bb20 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.686+0000 7f9dfca00640 10 client.4668 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059030 con 0x560b8405bb20 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 insert_trace from 2024-03-28T01:08:17.687254+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 features 0xffffffffffffffff 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 update_snap_trace len 70 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref now is 1 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:17.692 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 15 client.4668 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 15 client.4668 snapc 1=[] 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 put_snap_realm 0x1 0x7f9dcc00af00 2 -> 1 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 hrm is_target=1 is_dentry=0 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00aff0) - ctime 2024-03-28T01:08:16.651484+0000 mtime 2024-03-28T01:08:16.651484+0000 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 dir hash is 2 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 12 client.4668 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 9 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 12 client.4668 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=- has_dir_layout 0x7f9dcc00aff0) caps pAsLsXsFs 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref was 1 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref now is 2 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 15 client.4668 add_update_cap first one, opened snaprealm 0x7f9dcc00af00 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.693 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 add_update_cap calling signal_caps_inode 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 put_snap_realm 0x1 0x7f9dcc00af00 2 -> 1 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 handle_client_reply signalling caller 0x7ffe68269830 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 lat 0.000808 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 _ll_get 0x7f9dcc00aff0 0x1 -> 1 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 8 client.4668 _ll_create 0x1.head test_llreadvllwritevnegativeiovcountfile44790 0666 578, uid 0, gid 0 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 _do_lookup on #0x1/test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 choose_target_mds inode dir hash is 2 on test_llreadvllwritevnegativeiovcountfile44790 => 3167249749 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 choose_target_mds 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) is_hash=1 hash=3167249749 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 mds is 0 2024-03-28T01:08:17.694 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 encode_cap_releases enter (req: 0x560b840544a0, mds: 0) 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 send_request set sent_stamp to 2024-03-28T01:08:17.688105+0000 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 send_request client_request(unknown.0:2 lookup #0x1/test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.688081+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.688081+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa96d0 con 0x560b8405bb20 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 awaiting reply|forward|kick on 0x7ffe682695e0 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df0005ad0 con 0x560b8405bb20 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 insert_trace from 2024-03-28T01:08:17.688105+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 10 client.4668 insert_trace -- no trace 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 handle_client_reply signalling caller 0x7ffe682695e0 2024-03-28T01:08:17.695 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9dfca00640 20 client.4668 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 lat 0.000473 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 _do_lookup res is -2 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 _lookup 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) test_llreadvllwritevnegativeiovcountfile44790 = -2 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 may_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 _getattr mask As issued=1 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 3 client.4668 may_create 0x7f9dcc00aff0 = 0 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 8 client.4668 _create(0x1 test_llreadvllwritevnegativeiovcountfile44790, 0666) 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 get_or_create 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) name test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 15 open_dir 0x560b84054310 on 0x7f9dcc00aff0 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 15 client.4668 link dir 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 'test_llreadvllwritevnegativeiovcountfile44790' dn 0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] (new dn) 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 choose_target_mds inode dir hash is 2 on test_llreadvllwritevnegativeiovcountfile44790 => 3167249749 2024-03-28T01:08:17.696 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 choose_target_mds 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) is_hash=1 hash=3167249749 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 mds is 0 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 encode_cap_releases enter (req: 0x560b83fa96d0, mds: 0) 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 encode_dentry_release enter(dn:0x7f9dcc00abf0) 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0), req:0x560b83fa96d0 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 20 client.4668 send_request set sent_stamp to 2024-03-28T01:08:17.688650+0000 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 10 client.4668 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.688621+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.687+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.688621+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa9b00 con 0x560b8405bb20 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.688+0000 7f9e0324a480 20 client.4668 awaiting reply|forward|kick on 0x7ffe68269670 2024-03-28T01:08:17.697 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.688+0000 7f9dfca00640 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 866+0+0 (crc 0 0 0) 0x7f9df0019c40 con 0x560b8405bb20 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 20 client.4668 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 10 client.4668 insert_trace from 2024-03-28T01:08:17.688650+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 10 client.4668 features 0xffffffffffffffff 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 10 client.4668 update_snap_trace len 70 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref was 1 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref now is 2 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 10 client.4668 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 10 client.4668 hrm is_target=1 is_dentry=1 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 10 client.4668 update_inode_file_time 0x10000000005.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) - ctime 2024-03-28T01:08:17.688621+0000 mtime 2024-03-28T01:08:17.688621+0000 2024-03-28T01:08:17.703 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 12 client.4668 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.692+0000 7f9dfca00640 12 client.4668 add_update_inode adding 0x10000000005.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=- objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref was 2 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref now is 3 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 15 client.4668 add_update_cap first one, opened snaprealm 0x7f9dcc00af00 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 10 client.4668 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000005.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 10 client.4668 add_update_cap calling signal_caps_inode 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 12 client.4668 add_update_inode had 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) caps pAsLsXsFs 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 10 client.4668 update_inode_file_time 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:16.651484+0000 ctime=2024-03-28T01:08:16.651484+0000 change_attr=9 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) pAsLsXs ctime 2024-03-28T01:08:17.688621+0000 mtime 2024-03-28T01:08:17.688621+0000 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 dir hash is 2 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 12 client.4668 add_update_inode client inode change_attr: 9 , mds inodestat change_attr: 10 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 10 client.4668 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 10 client.4668 add_update_cap calling signal_caps_inode 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:17.704 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 12 client.4668 insert_dentry_inode 'test_llreadvllwritevnegativeiovcountfile44790' vino 0x10000000005.head in dir 0x1.head dn 0x7f9dcc00abf0 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 15 client.4668 link dir 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 'test_llreadvllwritevnegativeiovcountfile44790' to inode 0x7f9dcc00b5d0 dn 0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] (old dn) 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 link inode 0x7f9dcc00b5d0 parents now 0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 15 client.4668 update_dentry_lease 0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:17.688650+0000 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 put_snap_realm 0x1 0x7f9dcc00af00 3 -> 2 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 handle_client_reply signalling caller 0x7ffe68269670 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 delegated_inos: [] 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 make_request created ino 0x10000000005 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 make_request target is 0x10000000005.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.705 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 lat 0.005283 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000005.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000005.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 _create_fh 0x10000000005 mode 3 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 trim_cache size 1 max 16384 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 8 client.4668 create(#0x1/test_llreadvllwritevnegativeiovcountfile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 _ll_create created = 1 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 8 client.4668 _ll_create 0x1.head test_llreadvllwritevnegativeiovcountfile44790 0666 578 = 0 (0x560b83fa9b00 10000000005) 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 _ll_get 0x7f9dcc00b5d0 0x10000000005 -> 1 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 fill_statx on 0x10000000005 snap/devhead mode 0100666 mtime 2024-03-28T01:08:17.688621+0000 ctime 2024-03-28T01:08:17.688621+0000 change_attr 0 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.706 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 3 client.4668 ll_release (fh)0x560b83fa9b00 0x10000000005 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 8 client.4668 _release_fh 0x560b83fa9b00 mode 3 on 0x10000000005.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 _flush 0x10000000005.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 nothing to flush 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 check_caps on 0x10000000005.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) wanted - used - issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 delaying cap release 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 cap_delay_requeue on 0x10000000005.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 _release_fh 0x560b83fa9b00 on inode 0x10000000005.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) no async_err state 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 3 client.4668 ll_unlink 0x1.head test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 may_delete 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0); ; name test_llreadvllwritevnegativeiovcountfile44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:17.707 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 _getattr mask As issued=1 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 3 client.4668 may_delete 0x7f9dcc00aff0 = 0 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 8 client.4668 _unlink(0x1 test_llreadvllwritevnegativeiovcountfile44790 uid 0 gid 0) 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 get_or_create 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) name test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 _lookup have 0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 _lookup 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) test_llreadvllwritevnegativeiovcountfile44790 = 0x10000000005.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000005.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000005.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 choose_target_mds inode dir hash is 2 on test_llreadvllwritevnegativeiovcountfile44790 => 3167249749 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 choose_target_mds 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0) is_hash=1 hash=3167249749 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 choose_target_mds from dirfragtree hash 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 mds is 0 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:17.708 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 encode_cap_releases enter (req: 0x560b83fad260, mds: 0) 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 encode_inode_release enter(in:0x10000000005.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0), req:0x560b83fad260 mds:0, drop:LsxXx, unless:-, force:0) 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 encode_dentry_release enter(dn:0x7f9dcc00abf0) 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00aff0), req:0x560b83fad260 mds:0, drop:Fs, unless:Fx, force:1) 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 send_request set sent_stamp to 2024-03-28T01:08:17.694431+0000 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 10 client.4668 send_request client_request(unknown.0:4 unlink #0x1/test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.694371+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 unlink #0x1/test_llreadvllwritevnegativeiovcountfile44790 2024-03-28T01:08:17.694371+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fad590 con 0x560b8405bb20 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9e0324a480 20 client.4668 awaiting reply|forward|kick on 0x7ffe68269850 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.693+0000 7f9dfca00640 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(revoke ino 0x10000000005 1 seq 2 caps=pAsXsFsxcrwb dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:17.688621+0000 ctime 2024-03-28T01:08:17.688621+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0094040 con 0x560b8405bb20 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 5 client.4668 set_cap_epoch_barrier epoch = 20 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 mds.0 seq now 1 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 5 client.4668 handle_cap_grant on in 0x10000000005 mds.0 seq 2 caps now pAsXsFsxcrwb was pAsxXsFsxcrwb 2024-03-28T01:08:17.709 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 update_inode_file_time 0x10000000005.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) pAsxXsFsxcrwb ctime 2024-03-28T01:08:17.688621+0000 mtime 2024-03-28T01:08:17.688621+0000 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 revocation of Ax 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 check_caps on 0x10000000005.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) wanted - used - issued pAsXsFsxcrwb revoking Ax flags=1 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsFsxcrwb revoking Ax 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 completed revocation of Ax 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 send_cap 0x10000000005.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevnegativeiovcountfile44790"] 0x7f9dcc00b5d0) mds.0 seq 2 used - want - flush - retain pAsLsXsFs held pAsxXsFsxcrwb revoking Ax dropping Fxcrwb 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 15 client.4668 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000005 1 seq 2 caps=pAsXsFs dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:17.688621+0000 ctime 2024-03-28T01:08:17.688621+0000 change_attr 0) v12 -- 0x7f9dcc00bbb0 con 0x560b8405bb20 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.694+0000 7f9dfca00640 10 client.4668 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:4 = 0 (0) Success unsafe) v1 ==== 514+0+0 (crc 0 0 0) 0x7f9df0054bd0 con 0x560b8405bb20 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 handle_client_reply got a reply. Safe:0 tid 4 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 insert_trace from 2024-03-28T01:08:17.694431+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 features 0xffffffffffffffff 2024-03-28T01:08:17.710 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 update_snap_trace len 70 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref was 2 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 get_snap_realm 0x1 0x7f9dcc00af00, nref now is 3 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 hrm is_target=0 is_dentry=1 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 12 client.4668 add_update_inode had 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) caps pAsLsXs 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 update_inode_file_time 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=10 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) pAsLsXs ctime 2024-03-28T01:08:17.694371+0000 mtime 2024-03-28T01:08:17.694371+0000 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 dir hash is 2 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 12 client.4668 add_update_inode client inode change_attr: 10 , mds inodestat change_attr: 11 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 15 client.4668 unlink dir 0x7f9dcc00aff0 'test_llreadvllwritevnegativeiovcountfile44790' dn 0x7f9dcc00abf0 inode 0x7f9dcc00b5d0 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 unlink inode 0x7f9dcc00b5d0 parents now 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.711 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 put_snap_realm 0x1 0x7f9dcc00af00 3 -> 2 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 handle_client_reply signalling caller 0x7ffe68269850 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 handle_client_reply awaiting kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 20 client.4668 sendrecv kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 20 client.4668 lat 0.005002 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 20 client.4668 trim_cache size 1 max 16384 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 8 client.4668 unlink(#0x1/test_llreadvllwritevnegativeiovcountfile44790) = 0 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 2 client.4668 unmount 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 2 client.4668 unmounting 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83fad590 con 0x560b8405bb20 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 10 client.4668 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size was 1+2 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 20 client.4668 trim_cache size 1 max 16384 2024-03-28T01:08:17.712 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size still 1+2 2024-03-28T01:08:17.713 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.698+0000 7f9e0324a480 10 client.4668 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:17.714 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_caps(grant ino 0x10000000005 1 seq 3 caps=pAsLsXsFsxcrwb dirty=- wanted=Ls follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:17.688621+0000 ctime 2024-03-28T01:08:17.694371+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dcc00bbb0 con 0x560b8405bb20 2024-03-28T01:08:17.714 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 mds.0 seq now 2 2024-03-28T01:08:17.714 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 5 client.4668 handle_cap_grant on in 0x10000000005 mds.0 seq 3 caps now pAsLsXsFsxcrwb was pAsXsFs 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 update_inode_file_time 0x10000000005.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.688621+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) pAsXsFs ctime 2024-03-28T01:08:17.694371+0000 mtime 2024-03-28T01:08:17.688621+0000 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 max_size 4194304 -> 0 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 grant, new caps are LsFxcrwb 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 5 client.4668 _try_to_trim_inode in 0x10000000005.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size was 1+2 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 20 client.4668 trim_cache size 1 max 16384 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size still 1+2 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:4 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df0054dc0 con 0x560b8405bb20 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 20 client.4668 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 insert_trace from 2024-03-28T01:08:17.694431+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.715 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size was 1+2 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 20 client.4668 trim_cache size 1 max 16384 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 15 client.4668 trim_dentry unlinking dn test_llreadvllwritevnegativeiovcountfile44790 in dir 0x1 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 15 client.4668 unlink dir 0x7f9dcc00aff0 'test_llreadvllwritevnegativeiovcountfile44790' dn 0x7f9dcc00abf0 inode 0 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 15 client.4668 unlink removing 'test_llreadvllwritevnegativeiovcountfile44790' dn 0x7f9dcc00abf0 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 15 client.4668 close_dir dir 0x560b84054310 on 0x7f9dcc00aff0 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _ll_drop_pins 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 _ll_put 0x7f9dcc00b5d0 0x10000000005 1 -> 0 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 _ll_put 0x7f9dcc00aff0 0x1 2 -> 0 2024-03-28T01:08:17.716 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(grant ino 0x10000000005 1 seq 4 caps=pAsLsXsFsxcrwb dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:17.688621+0000 ctime 2024-03-28T01:08:17.694371+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081040 con 0x560b8405bb20 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 _release 0x10000000005.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _invalidate_inode_cache 0x10000000005.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _flush 0x10000000005.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 nothing to flush 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 _release 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _flush 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 nothing to flush 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.717 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x10000000005.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 1 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 1 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 flush_caps_sync 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 trim_cache size 0 max 16384 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) n = 12 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 15 client.4668 remove_cap last one, closing snaprealm 0x7f9dcc00af00 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_snap_realm 0x1 0x7f9dcc00af00 2 -> 1 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=- has_dir_layout 0x7f9dcc00aff0) 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _put_inode on 0x10000000005.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) n = 15 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 remove_cap mds.0 on 0x10000000005.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 15 client.4668 remove_cap last one, closing snaprealm 0x7f9dcc00af00 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 20 client.4668 put_snap_realm 0x1 0x7f9dcc00af00 1 -> 0 2024-03-28T01:08:17.718 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 10 client.4668 _put_inode deleting 0x10000000005.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:17.688621+0000 mtime=2024-03-28T01:08:17.688621+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=1 caps=- objectset[0x10000000005 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dcc00b5d0) 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 2 client.4668 _close_mds_session mds.0 seq 2 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 2) v7 -- 0x560b83fadaa0 con 0x560b8405bb20 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9e0324a480 2 client.4668 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 mds.0 seq now 3 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 3) v7 -- 0x7f9dcc00ace0 con 0x560b8405bb20 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 5 client.4668 handle_caps don't have vino 0x10000000005.head op 0, immediately releasing 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_cap_release(3) v2 -- 0x560b83fad8f0 con 0x560b8405bb20 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size was 0+0 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 20 client.4668 trim_cache size 0 max 16384 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.704+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size still 0+0 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83fad590 con 0x560b8405bb20 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 10 client.4668 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:17.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 5 client.4668 _closed_mds_session mds.0 seq 3 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 1 -- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 msgr2=0x560b83f2e280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2e280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 10 client.4668 remove_session_caps mds.0 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 10 client.4668 kick_requests_closed for mds.0 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size was 0+0 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 20 client.4668 trim_cache size 0 max 16384 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9dfca00640 10 client.4668 unmounting: trim pass, size still 0+0 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 2 client.4668 unmounted. 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 client.4668 shutdown 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fadf60 msgr2=0x560b83fb3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x560b8405a630 tx=0x7f9de8051040 comp rx=0 tx=0).stop 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 shutdown_connections 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8405bb20 0x560b83f2e280 unknown :-1 s=CLOSED pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fadf60 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4143779741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83ee5730 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.709+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 >> 172.21.15.33:0/4143779741 conn(0x560b83f2d970 msgr2=0x560b83f29dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.710+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 shutdown_connections 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.710+0000 7f9e0324a480 1 -- 172.21.15.33:0/4143779741 wait complete. 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.710+0000 7f9e0324a480 20 client.4668 trim_cache size 0 max 16384 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritevNegativeIOVCount (1049 ms) 2024-03-28T01:08:17.721 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritevZeroBytes 2024-03-28T01:08:17.725 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.711+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:17.725 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.713+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:17.725 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.725 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.725 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.725 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0067b0 con 0x560b83ee5730 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f28d60 con 0x560b8405bb20 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f860 con 0x560b83fb16a0 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49514/0 (socket says 172.21.15.33:49514) 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.714+0000 7f9dffc00640 1 -- 172.21.15.33:0/4229418602 learned_addr learned my addr 172.21.15.33:0/4229418602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dff200640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9e00600640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dffc00640 1 -- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dffc00640 1 -- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dffc00640 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b840618c0 con 0x560b83ee5730 2024-03-28T01:08:17.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9e00600640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9df0002010 tx=0x560b84061400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b29de956701874a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.715+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0099040 con 0x560b83ee5730 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df0005ad0 con 0x560b83ee5730 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 10 client.4671 Subscribing to map 'mdsmap' 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83faa550 con 0x560b83ee5730 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9df5600640 20 client.4671 tick 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9df5600640 20 client.4671 collect_and_send_metrics 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9df5600640 20 client.4671 collect_and_send_global_metrics 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9df5600640 5 client.4671 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9df5600640 20 client.4671 trim_cache size 0 max 16384 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9df5600640 20 client.4671 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b83fad8f0 con 0x560b83ee5730 2024-03-28T01:08:17.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0091040 con 0x560b83ee5730 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9dfde00640 1 client.4671 handle_mds_map epoch 8 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 20 client.4671 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 10 client.4671 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 20 client.4671 mds is 0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 10 client.4671 _open_mds_session mds.0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406ade0 0x560b83f2e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b8406ade0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.716+0000 7f9e0324a480 10 client.4671 waiting for session to mds.0 to open 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.717+0000 7f9dff200640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406ade0 0x560b83f2e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.717+0000 7f9dff200640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406ade0 0x560b83f2e280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0745357e872df9 server_cookie=1585e00cba59c27d in_seq=0 out_seq=0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.717+0000 7f9dfde00640 10 client.4671 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b8406ade0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 10 client.4671 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 10 client.4671 renew_caps mds.0 2024-03-28T01:08:17.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dcc006230 con 0x560b8406ade0 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 10 client.4671 connect_mds_targets for mds.0 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 10 client.4671 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 20 client.4671 mds is 0 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 10 client.4671 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 20 client.4671 encode_cap_releases enter (req: 0x560b83fa9c60, mds: 0) 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 20 client.4671 send_request set sent_stamp to 2024-03-28T01:08:17.721970+0000 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 10 client.4671 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:17.717361+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:17.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:17.717361+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa95d0 con 0x560b8406ade0 2024-03-28T01:08:17.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9e0324a480 20 client.4671 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:17.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dcc006230 con 0x560b8406ade0 2024-03-28T01:08:17.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 10 client.4671 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:17.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9dc4001460 con 0x560b8406ade0 2024-03-28T01:08:17.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.721+0000 7f9dfde00640 20 client.4671 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:17.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 insert_trace from 2024-03-28T01:08:17.721970+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:17.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 features 0xffffffffffffffff 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 update_snap_trace len 70 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 get_snap_realm 0x1 0x7f9dcc00aff0, nref now is 1 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 15 client.4671 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 15 client.4671 snapc 1=[] 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 put_snap_realm 0x1 0x7f9dcc00aff0 2 -> 1 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 hrm is_target=1 is_dentry=0 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dcc00b0e0) - ctime 2024-03-28T01:08:17.694371+0000 mtime 2024-03-28T01:08:17.694371+0000 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 dir hash is 2 2024-03-28T01:08:17.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 12 client.4671 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 11 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 12 client.4671 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=- has_dir_layout 0x7f9dcc00b0e0) caps pAsLsXsFs 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 get_snap_realm 0x1 0x7f9dcc00aff0, nref was 1 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 get_snap_realm 0x1 0x7f9dcc00aff0, nref now is 2 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 15 client.4671 add_update_cap first one, opened snaprealm 0x7f9dcc00aff0 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 add_update_cap calling signal_caps_inode 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 put_snap_realm 0x1 0x7f9dcc00aff0 2 -> 1 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 lat 0.000781 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 _ll_get 0x7f9dcc00b0e0 0x1 -> 1 2024-03-28T01:08:17.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 2 client.4671 unmount 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 2 client.4671 unmounting 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83fa95d0 con 0x560b8406ade0 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 10 client.4671 _ll_drop_pins 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 _ll_put 0x7f9dcc00b0e0 0x1 1 -> 0 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 10 client.4671 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 10 client.4671 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 10 client.4671 nothing to flush 2024-03-28T01:08:17.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 10 client.4671 flush_caps_sync 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 10 client.4671 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 20 client.4671 trim_cache size 0 max 16384 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 10 client.4671 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 6 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 2 client.4671 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 1 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 unmounting: trim pass, size was 0+1 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 20 client.4671 trim_cache size 0 max 16384 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9dfde00640 10 client.4671 unmounting: trim pass, size still 0+1 2024-03-28T01:08:17.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:17.722+0000 7f9e0324a480 2 client.4671 cache still has 0+1 items, waiting (for caps to release?) 2024-03-28T01:08:18.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 20 client.4671 tick 2024-03-28T01:08:18.719 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 10 client.4671 renew_caps() 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 15 client.4671 renew_caps requesting from mds.0 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 10 client.4671 renew_caps mds.0 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 2) v7 -- 0x7f9dc4001b30 con 0x560b8406ade0 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 20 client.4671 collect_and_send_metrics 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 20 client.4671 collect_and_send_global_metrics 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_metrics [client_metric_type: READ_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: WRITE_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: METADATA_LATENCY latency: 0.000781, avg_latency: 0.000781, sq_sum: 0, count=1][client_metric_type: CAP_INFO cap_hits: 0 cap_misses: 0 num_caps: 0][client_metric_type: DENTRY_LEASE dlease_hits: 0 dlease_misses: 0 num_dentries: 0][client_metric_type: OPENED_FILES opened_files: 0 total_inodes: 1][client_metric_type: PINNED_ICAPS pinned_icaps: 1 total_inodes: 1][client_metric_type: OPENED_INODES opened_inodes: 0 total_inodes: 1][client_metric_type: READ_IO_SIZES total_ops: 0 total_size: 0][client_metric_type: WRITE_IO_SIZES total_ops: 0 total_size: 0] v1 -- 0x7f9dc4002f10 con 0x560b8406ade0 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 10 client.4671 _put_inode on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) n = 2 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 10 client.4671 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dcc00b0e0) 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 15 client.4671 remove_cap last one, closing snaprealm 0x7f9dcc00aff0 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 20 client.4671 put_snap_realm 0x1 0x7f9dcc00aff0 1 -> 0 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 10 client.4671 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=- has_dir_layout 0x7f9dcc00b0e0) 2024-03-28T01:08:18.720 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 20 client.4671 trim_cache size 0 max 16384 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9df5600640 20 client.4671 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9e0324a480 2 client.4671 _close_mds_session mds.0 seq 0 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83fa97f0 con 0x560b8406ade0 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.716+0000 7f9e0324a480 2 client.4671 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.717+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc4001b30 con 0x560b8406ade0 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.717+0000 7f9dfde00640 10 client.4671 handle_client_session client_session(renewcaps seq 2) v1 from mds.0 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.717+0000 7f9dfde00640 10 client.4671 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.717+0000 7f9dfde00640 20 client.4671 trim_cache size 0 max 16384 2024-03-28T01:08:18.721 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.717+0000 7f9dfde00640 10 client.4671 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83fa95d0 con 0x560b8406ade0 2024-03-28T01:08:18.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 10 client.4671 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:18.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 5 client.4671 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:18.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 1 -- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406ade0 msgr2=0x560b83f2e280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406ade0 0x560b83f2e280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.726 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 10 client.4671 remove_session_caps mds.0 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 10 client.4671 kick_requests_closed for mds.0 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 10 client.4671 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 20 client.4671 trim_cache size 0 max 16384 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9dfde00640 10 client.4671 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9e0324a480 2 client.4671 unmounted. 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.723+0000 7f9e0324a480 1 client.4671 shutdown 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 msgr2=0x560b83fb3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9df0002010 tx=0x560b84061400 comp rx=0 tx=0).stop 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 shutdown_connections 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406ade0 0x560b83f2e280 unknown :-1 s=CLOSED pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8405bb20 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4229418602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee5730 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.727 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 >> 172.21.15.33:0/4229418602 conn(0x560b83f2d970 msgr2=0x560b83f29dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:18.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 shutdown_connections 2024-03-28T01:08:18.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 1 -- 172.21.15.33:0/4229418602 wait complete. 2024-03-28T01:08:18.728 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.724+0000 7f9e0324a480 20 client.4671 trim_cache size 0 max 16384 2024-03-28T01:08:18.729 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.726+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:18.730 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.728+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:18.731 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.729+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.729+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.729+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8406ade0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.729+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc007eb0 con 0x560b8405bb20 2024-03-28T01:08:18.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.729+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f610 con 0x560b83fb16a0 2024-03-28T01:08:18.732 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.729+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8406cbf0 con 0x560b8406ade0 2024-03-28T01:08:18.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.729+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b83fb3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49524/0 (socket says 172.21.15.33:49524) 2024-03-28T01:08:18.733 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dff200640 1 -- 172.21.15.33:0/274442347 learned_addr learned my addr 172.21.15.33:0/274442347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9e00600640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8406ade0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dffc00640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dff200640 1 -- 172.21.15.33:0/274442347 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8406ade0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dff200640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b8406ade0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dff200640 1 -- 172.21.15.33:0/274442347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dff200640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9dff200640 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b84054450 con 0x560b8405bb20 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9e0324a480 10 client.4674 Subscribing to map 'mdsmap' 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.730+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mdsmap=0+}) v3 -- 0x560b83fa9220 con 0x560b8405bb20 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9df6000640 20 client.4674 tick 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9df6000640 20 client.4674 collect_and_send_metrics 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9df6000640 20 client.4674 collect_and_send_global_metrics 2024-03-28T01:08:18.734 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9df6000640 5 client.4674 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:18.735 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9df6000640 20 client.4674 trim_cache size 0 max 16384 2024-03-28T01:08:18.735 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9df6000640 20 client.4674 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:18.735 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9dff200640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b83fb3f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000c00 tx=0x560b83eff430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b5995c91fa0bd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:18.735 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9dfca00640 10 client.4674 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4002f10 con 0x560b8405bb20 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.731+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de8000c30 con 0x560b8405bb20 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025040 con 0x560b8405bb20 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9dfca00640 1 client.4674 handle_mds_map epoch 8 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9e0324a480 20 client.4674 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9e0324a480 10 client.4674 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9e0324a480 20 client.4674 mds is 0 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9e0324a480 10 client.4674 _open_mds_session mds.0 2024-03-28T01:08:18.736 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9e0324a480 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fad220 0x560b83fa9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.737 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b83fad220 2024-03-28T01:08:18.737 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9e0324a480 10 client.4674 waiting for session to mds.0 to open 2024-03-28T01:08:18.737 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.732+0000 7f9dffc00640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fad220 0x560b83fa9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.737 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.733+0000 7f9dffc00640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fad220 0x560b83fa9590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173c87013095246c server_cookie=6532687a0c0a5150 in_seq=0 out_seq=0 2024-03-28T01:08:18.737 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.733+0000 7f9dfca00640 10 client.4674 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:18.741 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b83fad220 2024-03-28T01:08:18.741 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:18.741 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 renew_caps mds.0 2024-03-28T01:08:18.741 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc4009c70 con 0x560b83fad220 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 connect_mds_targets for mds.0 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 10 client.4674 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 20 client.4674 mds is 0 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 10 client.4674 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 20 client.4674 encode_cap_releases enter (req: 0x560b83fb44b0, mds: 0) 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 20 client.4674 send_request set sent_stamp to 2024-03-28T01:08:18.737977+0000 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 10 client.4674 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.732868+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.732868+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb47e0 con 0x560b83fad220 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 20 client.4674 awaiting reply|forward|kick on 0x7ffe68269840 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc4009c70 con 0x560b83fad220 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059070 con 0x560b83fad220 2024-03-28T01:08:18.742 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 insert_trace from 2024-03-28T01:08:18.737977+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 features 0xffffffffffffffff 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 update_snap_trace len 70 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref now is 1 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 15 client.4674 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 15 client.4674 snapc 1=[] 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 put_snap_realm 0x1 0x7f9dc4009e90 2 -> 1 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 hrm is_target=1 is_dentry=0 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc4009f80) - ctime 2024-03-28T01:08:17.694371+0000 mtime 2024-03-28T01:08:17.694371+0000 2024-03-28T01:08:18.743 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 dir hash is 2 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 12 client.4674 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 11 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 12 client.4674 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=- has_dir_layout 0x7f9dc4009f80) caps pAsLsXsFs 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref was 1 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref now is 2 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 15 client.4674 add_update_cap first one, opened snaprealm 0x7f9dc4009e90 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 10 client.4674 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 put_snap_realm 0x1 0x7f9dc4009e90 2 -> 1 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 handle_client_reply signalling caller 0x7ffe68269840 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9dfca00640 20 client.4674 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.737+0000 7f9e0324a480 20 client.4674 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 lat 0.000694 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 _ll_get 0x7f9dc4009f80 0x1 -> 1 2024-03-28T01:08:18.744 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 8 client.4674 _ll_create 0x1.head test_llreadvllwritevzerobytesfile44790 0666 578, uid 0, gid 0 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 _do_lookup on #0x1/test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 choose_target_mds inode dir hash is 2 on test_llreadvllwritevzerobytesfile44790 => 48833935 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 choose_target_mds 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) is_hash=1 hash=48833935 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 mds is 0 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 encode_cap_releases enter (req: 0x560b83fb47e0, mds: 0) 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 send_request set sent_stamp to 2024-03-28T01:08:18.738786+0000 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 send_request client_request(unknown.0:2 lookup #0x1/test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.738707+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.738707+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84053140 con 0x560b83fad220 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 awaiting reply|forward|kick on 0x7ffe682695f0 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.745 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df0019c40 con 0x560b83fad220 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 20 client.4674 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 10 client.4674 insert_trace from 2024-03-28T01:08:18.738786+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 10 client.4674 insert_trace -- no trace 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 20 client.4674 handle_client_reply signalling caller 0x7ffe682695f0 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 20 client.4674 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 lat 0.000455 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 _do_lookup res is -2 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 _lookup 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) test_llreadvllwritevzerobytesfile44790 = -2 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 may_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 _getattr mask As issued=1 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 3 client.4674 may_create 0x7f9dc4009f80 = 0 2024-03-28T01:08:18.746 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 8 client.4674 _create(0x1 test_llreadvllwritevzerobytesfile44790, 0666) 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 get_or_create 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) name test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 15 open_dir 0x560b83fa9220 on 0x7f9dc4009f80 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 15 client.4674 link dir 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 'test_llreadvllwritevzerobytesfile44790' dn 0x1.head["test_llreadvllwritevzerobytesfile44790"] (new dn) 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 choose_target_mds inode dir hash is 2 on test_llreadvllwritevzerobytesfile44790 => 48833935 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 choose_target_mds 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) is_hash=1 hash=48833935 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 mds is 0 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 encode_cap_releases enter (req: 0x560b84053140, mds: 0) 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 encode_dentry_release enter(dn:0x7f9dcc00af00) 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80), req:0x560b84053140 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 send_request set sent_stamp to 2024-03-28T01:08:18.739311+0000 2024-03-28T01:08:18.747 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 10 client.4674 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.739282+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.748 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.739282+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84053530 con 0x560b83fad220 2024-03-28T01:08:18.748 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9e0324a480 20 client.4674 awaiting reply|forward|kick on 0x7ffe68269680 2024-03-28T01:08:18.748 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.738+0000 7f9dfca00640 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 859+0+0 (crc 0 0 0) 0x7f9df0005a80 con 0x560b83fad220 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 insert_trace from 2024-03-28T01:08:18.739311+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 features 0xffffffffffffffff 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 update_snap_trace len 70 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref was 1 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref now is 2 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 hrm is_target=1 is_dentry=1 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 update_inode_file_time 0x10000000006.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) - ctime 2024-03-28T01:08:18.739282+0000 mtime 2024-03-28T01:08:18.739282+0000 2024-03-28T01:08:18.750 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 12 client.4674 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 12 client.4674 add_update_inode adding 0x10000000006.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=- objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref was 2 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref now is 3 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 15 client.4674 add_update_cap first one, opened snaprealm 0x7f9dc4009e90 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000006.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 12 client.4674 add_update_inode had 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) caps pAsLsXsFs 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 update_inode_file_time 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:17.694371+0000 ctime=2024-03-28T01:08:17.694371+0000 change_attr=11 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) pAsLsXs ctime 2024-03-28T01:08:18.739282+0000 mtime 2024-03-28T01:08:18.739282+0000 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 dir hash is 2 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 12 client.4674 add_update_inode client inode change_attr: 11 , mds inodestat change_attr: 12 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 10 client.4674 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.751 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 12 client.4674 insert_dentry_inode 'test_llreadvllwritevzerobytesfile44790' vino 0x10000000006.head in dir 0x1.head dn 0x7f9dcc00af00 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 15 client.4674 link dir 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 'test_llreadvllwritevzerobytesfile44790' to inode 0x7f9dc400a560 dn 0x1.head["test_llreadvllwritevzerobytesfile44790"] (old dn) 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 link inode 0x7f9dc400a560 parents now 0x1.head["test_llreadvllwritevzerobytesfile44790"] 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 15 client.4674 update_dentry_lease 0x1.head["test_llreadvllwritevzerobytesfile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:18.739311+0000 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 put_snap_realm 0x1 0x7f9dc4009e90 3 -> 2 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 handle_client_reply signalling caller 0x7ffe68269680 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 10 client.4674 delegated_inos: [] 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 10 client.4674 make_request created ino 0x10000000006 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 make_request target is 0x10000000006.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 lat 0.005099 2024-03-28T01:08:18.752 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000006.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000006.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 10 client.4674 _create_fh 0x10000000006 mode 3 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 trim_cache size 1 max 16384 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 8 client.4674 create(#0x1/test_llreadvllwritevzerobytesfile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 _ll_create created = 1 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 8 client.4674 _ll_create 0x1.head test_llreadvllwritevzerobytesfile44790 0666 578 = 0 (0x560b83fad640 10000000006) 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 _ll_get 0x7f9dc400a560 0x10000000006 -> 1 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 10 client.4674 fill_statx on 0x10000000006 snap/devhead mode 0100666 mtime 2024-03-28T01:08:18.739282+0000 ctime 2024-03-28T01:08:18.739282+0000 change_attr 0 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 3 client.4674 pwritev(0x560b83fad640, "...", 0, 0) = -22 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x560b84053d70 0x560b83fa52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.753 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] -- osd_op(unknown.0.0:1 3.25 3:a5feb6f3:::10000000006.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x560b83fa5820 con 0x560b84053d70 2024-03-28T01:08:18.754 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] -- osd_op(unknown.0.0:2 3.25 3:a5feb6f3:::10000000006.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) v9 -- 0x560b83fa6090 con 0x560b84053d70 2024-03-28T01:08:18.754 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.754 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.743+0000 7f9dfca00640 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.754 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.744+0000 7f9e00600640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x560b84053d70 0x560b83fa52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.754 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.745+0000 7f9e00600640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x560b84053d70 0x560b83fa52e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=d3ef2ce8a722a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:18.754 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.745+0000 7f9dfca00640 10 client.4674 ms_handle_connect on v2:172.21.15.33:6816/4011280304 2024-03-28T01:08:18.754 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.746+0000 7f9e00600640 1 -- 172.21.15.33:0/274442347 <== osd.1 v2:172.21.15.33:6816/4011280304 1 ==== osd_op_reply(1 10000000006.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8029040 con 0x560b84053d70 2024-03-28T01:08:18.756 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e00600640 1 -- 172.21.15.33:0/274442347 <== osd.1 v2:172.21.15.33:6816/4011280304 2 ==== osd_op_reply(2 10000000006.00000000 [create] v22'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8029040 con 0x560b84053d70 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 get_caps 0x10000000006.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) have pAsxLsXsxFsxcrwb need Fr want Fc revoking - 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 _read_async 0x10000000006.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={2048=1} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 0~0 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 3 client.4674 preadv(0x560b83fad640, 0) = 0 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 3 client.4674 ll_release (fh)0x560b83fad640 0x10000000006 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 8 client.4674 _release_fh 0x560b83fad640 mode 3 on 0x10000000006.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={2048=0} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 _flush 0x10000000006.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 nothing to flush 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 check_caps on 0x10000000006.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) wanted - used - issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 delaying cap release 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 cap_delay_requeue on 0x10000000006.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.757 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 _release_fh 0x560b83fad640 on inode 0x10000000006.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) no async_err state 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 3 client.4674 ll_unlink 0x1.head test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 may_delete 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80); ; name test_llreadvllwritevzerobytesfile44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 _getattr mask As issued=1 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 3 client.4674 may_delete 0x7f9dc4009f80 = 0 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 8 client.4674 _unlink(0x1 test_llreadvllwritevzerobytesfile44790 uid 0 gid 0) 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 get_or_create 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) name test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 _lookup have 0x1.head["test_llreadvllwritevzerobytesfile44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 _lookup 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) test_llreadvllwritevzerobytesfile44790 = 0x10000000006.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000006.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000006.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.758 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 choose_target_mds inode dir hash is 2 on test_llreadvllwritevzerobytesfile44790 => 48833935 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 choose_target_mds 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80) is_hash=1 hash=48833935 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 choose_target_mds from dirfragtree hash 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 mds is 0 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 encode_cap_releases enter (req: 0x560b84053530, mds: 0) 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 encode_inode_release enter(in:0x10000000006.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560), req:0x560b84053530 mds:0, drop:LsxXx, unless:-, force:0) 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 encode_dentry_release enter(dn:0x7f9dcc00af00) 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc4009f80), req:0x560b84053530 mds:0, drop:Fs, unless:Fx, force:1) 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 send_request set sent_stamp to 2024-03-28T01:08:18.750188+0000 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 10 client.4674 send_request client_request(unknown.0:4 unlink #0x1/test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.750146+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 unlink #0x1/test_llreadvllwritevzerobytesfile44790 2024-03-28T01:08:18.750146+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa5c10 con 0x560b83fad220 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.749+0000 7f9e0324a480 20 client.4674 awaiting reply|forward|kick on 0x7ffe68269860 2024-03-28T01:08:18.759 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(revoke ino 0x10000000006 1 seq 2 caps=pAsXsFsxcrwb dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.739282+0000 ctime 2024-03-28T01:08:18.739282+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081070 con 0x560b83fad220 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 5 client.4674 set_cap_epoch_barrier epoch = 20 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 mds.0 seq now 1 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 5 client.4674 handle_cap_grant on in 0x10000000006 mds.0 seq 2 caps now pAsXsFsxcrwb was pAsxXsFsxcrwb 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 update_inode_file_time 0x10000000006.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) pAsxXsFsxcrwb ctime 2024-03-28T01:08:18.739282+0000 mtime 2024-03-28T01:08:18.739282+0000 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 revocation of Ax 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 check_caps on 0x10000000006.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) wanted - used - issued pAsXsFsxcrwb revoking Ax flags=1 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsFsxcrwb revoking Ax 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 completed revocation of Ax 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 send_cap 0x10000000006.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevzerobytesfile44790"] 0x7f9dc400a560) mds.0 seq 2 used - want - flush - retain pAsLsXsFs held pAsxXsFsxcrwb revoking Ax dropping Fxcrwb 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 15 client.4674 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000006 1 seq 2 caps=pAsXsFs dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.739282+0000 ctime 2024-03-28T01:08:18.739282+0000 change_attr 0) v12 -- 0x7f9dc400ab40 con 0x560b83fad220 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.750+0000 7f9dfca00640 10 client.4674 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:4 = 0 (0) Success unsafe) v1 ==== 507+0+0 (crc 0 0 0) 0x7f9df00548c0 con 0x560b83fad220 2024-03-28T01:08:18.760 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 handle_client_reply got a reply. Safe:0 tid 4 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 insert_trace from 2024-03-28T01:08:18.750188+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 features 0xffffffffffffffff 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 update_snap_trace len 70 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref was 2 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 get_snap_realm 0x1 0x7f9dc4009e90, nref now is 3 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 hrm is_target=0 is_dentry=1 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 12 client.4674 add_update_inode had 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) caps pAsLsXs 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 update_inode_file_time 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=12 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) pAsLsXs ctime 2024-03-28T01:08:18.750146+0000 mtime 2024-03-28T01:08:18.750146+0000 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 dir hash is 2 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 12 client.4674 add_update_inode client inode change_attr: 12 , mds inodestat change_attr: 13 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:18.761 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 15 client.4674 unlink dir 0x7f9dc4009f80 'test_llreadvllwritevzerobytesfile44790' dn 0x7f9dcc00af00 inode 0x7f9dc400a560 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 unlink inode 0x7f9dc400a560 parents now 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 put_snap_realm 0x1 0x7f9dc4009e90 3 -> 2 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 handle_client_reply signalling caller 0x7ffe68269860 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 handle_client_reply awaiting kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 20 client.4674 sendrecv kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 20 client.4674 lat 0.002263 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 20 client.4674 trim_cache size 1 max 16384 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 8 client.4674 unlink(#0x1/test_llreadvllwritevzerobytesfile44790) = 0 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 2 client.4674 unmount 2024-03-28T01:08:18.762 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 2 client.4674 unmounting 2024-03-28T01:08:18.763 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83fa5c10 con 0x560b83fad220 2024-03-28T01:08:18.763 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 10 client.4674 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:18.763 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size was 1+2 2024-03-28T01:08:18.763 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 20 client.4674 trim_cache size 1 max 16384 2024-03-28T01:08:18.763 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size still 1+2 2024-03-28T01:08:18.763 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.751+0000 7f9e0324a480 10 client.4674 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:18.766 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_caps(grant ino 0x10000000006 1 seq 3 caps=pAsLsXsFsxcrwb dirty=- wanted=Ls follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.739282+0000 ctime 2024-03-28T01:08:18.750146+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400ab40 con 0x560b83fad220 2024-03-28T01:08:18.766 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 mds.0 seq now 2 2024-03-28T01:08:18.766 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 5 client.4674 handle_cap_grant on in 0x10000000006 mds.0 seq 3 caps now pAsLsXsFsxcrwb was pAsXsFs 2024-03-28T01:08:18.766 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 update_inode_file_time 0x10000000006.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/4194304 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.739282+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) pAsXsFs ctime 2024-03-28T01:08:18.750146+0000 mtime 2024-03-28T01:08:18.739282+0000 2024-03-28T01:08:18.766 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 max_size 4194304 -> 0 2024-03-28T01:08:18.766 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 grant, new caps are LsFxcrwb 2024-03-28T01:08:18.766 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 5 client.4674 _try_to_trim_inode in 0x10000000006.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size was 1+2 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 20 client.4674 trim_cache size 1 max 16384 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size still 1+2 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:4 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df00638a0 con 0x560b83fad220 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 20 client.4674 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 insert_trace from 2024-03-28T01:08:18.750188+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size was 1+2 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 20 client.4674 trim_cache size 1 max 16384 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 15 client.4674 trim_dentry unlinking dn test_llreadvllwritevzerobytesfile44790 in dir 0x1 2024-03-28T01:08:18.767 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 15 client.4674 unlink dir 0x7f9dc4009f80 'test_llreadvllwritevzerobytesfile44790' dn 0x7f9dcc00af00 inode 0 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 15 client.4674 unlink removing 'test_llreadvllwritevzerobytesfile44790' dn 0x7f9dcc00af00 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 15 client.4674 close_dir dir 0x560b83fa9220 on 0x7f9dc4009f80 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _ll_drop_pins 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 _ll_put 0x7f9dc400a560 0x10000000006 1 -> 0 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 _ll_put 0x7f9dc4009f80 0x1 2 -> 0 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.768 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 _release 0x10000000006.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _invalidate_inode_cache 0x10000000006.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0,2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _flush 0x10000000006.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 nothing to flush 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 _release 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _flush 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 nothing to flush 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x10000000006.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 1 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 1 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 flush_caps_sync 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:18.769 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 trim_cache size 0 max 16384 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) n = 12 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 15 client.4674 remove_cap last one, closing snaprealm 0x7f9dc4009e90 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_snap_realm 0x1 0x7f9dc4009e90 2 -> 1 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=- has_dir_layout 0x7f9dc4009f80) 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _put_inode on 0x10000000006.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0,2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) n = 15 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(grant ino 0x10000000006 1 seq 4 caps=pAsLsXsFsxcrwb dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.739282+0000 ctime 2024-03-28T01:08:18.750146+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0094040 con 0x560b83fad220 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 remove_cap mds.0 on 0x10000000006.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0,2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 15 client.4674 remove_cap last one, closing snaprealm 0x7f9dc4009e90 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 20 client.4674 put_snap_realm 0x1 0x7f9dc4009e90 1 -> 0 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 10 client.4674 _put_inode deleting 0x10000000006.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0,2048=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.739282+0000 mtime=2024-03-28T01:08:18.739282+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=1 caps=- objectset[0x10000000006 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400a560) 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 2 client.4674 _close_mds_session mds.0 seq 2 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 2) v7 -- 0x560b84053a70 con 0x560b83fad220 2024-03-28T01:08:18.770 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9e0324a480 2 client.4674 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 mds.0 seq now 3 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 3) v7 -- 0x7f9dc4009c70 con 0x560b83fad220 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 5 client.4674 handle_caps don't have vino 0x10000000006.head op 0, immediately releasing 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_cap_release(3) v2 -- 0x560b840538c0 con 0x560b83fad220 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 20 client.4674 trim_cache size 0 max 16384 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.757+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83fa5c10 con 0x560b83fad220 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 10 client.4674 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 5 client.4674 _closed_mds_session mds.0 seq 3 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 1 -- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fad220 msgr2=0x560b83fa9590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fad220 0x560b83fa9590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 10 client.4674 remove_session_caps mds.0 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 10 client.4674 kick_requests_closed for mds.0 2024-03-28T01:08:18.771 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 20 client.4674 trim_cache size 0 max 16384 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9dfca00640 10 client.4674 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 2 client.4674 unmounted. 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 1 client.4674 shutdown 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x560b84053d70 msgr2=0x560b83fa52e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x560b84053d70 0x560b83fa52e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 msgr2=0x560b83fb3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 1 --2- 172.21.15.33:0/274442347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8405bb20 0x560b83fb3f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000c00 tx=0x560b83eff430 comp rx=0 tx=0).stop 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e00600640 1 -- 172.21.15.33:0/274442347 reap_dead start 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 shutdown_connections 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.762+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 >> 172.21.15.33:0/274442347 conn(0x560b83f2d970 msgr2=0x560b83f29dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.763+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 shutdown_connections 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.763+0000 7f9e0324a480 1 -- 172.21.15.33:0/274442347 wait complete. 2024-03-28T01:08:18.772 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.763+0000 7f9e0324a480 20 client.4674 trim_cache size 0 max 16384 2024-03-28T01:08:18.773 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritevZeroBytes (1053 ms) 2024-03-28T01:08:18.773 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.LlreadvLlwritevInvalidFileHandle 2024-03-28T01:08:18.777 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.764+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:18.777 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.765+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:18.777 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.765+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.765+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f2c6a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.765+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.765+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40085c0 con 0x560b83f2c6a0 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.765+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbb2a0 con 0x560b83fb16a0 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.765+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbede0 con 0x560b83fad220 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.766+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f2c6a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.766+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f2c6a0 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49526/0 (socket says 172.21.15.33:49526) 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.766+0000 7f9dff200640 1 -- 172.21.15.33:0/967273025 learned_addr learned my addr 172.21.15.33:0/967273025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.766+0000 7f9dffc00640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.766+0000 7f9e00600640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dff200640 1 -- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dff200640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.778 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dff200640 1 -- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dff200640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dff200640 1 -- 172.21.15.33:0/967273025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83ee4640 con 0x560b83f2c6a0 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9e00600640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9e0324a480 10 client.4677 Subscribing to map 'mdsmap' 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mdsmap=0+}) v3 -- 0x560b83f2b610 con 0x560b83f2c6a0 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dff200640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f2c6a0 0x560b84059ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x560b8404f5e0 tx=0x560b8406b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341dc4824c2a4dfb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dfde00640 10 client.4677 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9df5600640 20 client.4677 tick 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9df5600640 20 client.4677 collect_and_send_metrics 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9df5600640 20 client.4677 collect_and_send_global_metrics 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9df5600640 5 client.4677 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9df5600640 20 client.4677 trim_cache size 0 max 16384 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9df5600640 20 client.4677 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:18.779 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.767+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8065040 con 0x560b83f2c6a0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de8000bc0 con 0x560b83f2c6a0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025040 con 0x560b83f2c6a0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9dfde00640 1 client.4677 handle_mds_map epoch 8 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9e0324a480 20 client.4677 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9e0324a480 10 client.4677 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9e0324a480 20 client.4677 mds is 0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9e0324a480 10 client.4677 _open_mds_session mds.0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9e0324a480 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fa9590 0x560b83fb44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b84061370 con 0x560b83fa9590 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9e0324a480 10 client.4677 waiting for session to mds.0 to open 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.768+0000 7f9dffc00640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fa9590 0x560b83fb44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.769+0000 7f9dffc00640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fa9590 0x560b83fb44b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5d44c5eb29f44e server_cookie=9364bc9aa7a3f6e4 in_seq=0 out_seq=0 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.769+0000 7f9dfde00640 10 client.4677 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:18.780 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b84061370 con 0x560b83fa9590 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 renew_caps mds.0 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a370 con 0x560b83fa9590 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 connect_mds_targets for mds.0 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 10 client.4677 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 mds is 0 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 10 client.4677 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 encode_cap_releases enter (req: 0x560b84055260, mds: 0) 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 send_request set sent_stamp to 2024-03-28T01:08:18.773914+0000 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 10 client.4677 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.769109+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.769109+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f29260 con 0x560b83fa9590 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:18.781 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a370 con 0x560b83fa9590 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0019c40 con 0x560b83fa9590 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 insert_trace from 2024-03-28T01:08:18.773914+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 features 0xffffffffffffffff 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 update_snap_trace len 70 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 get_snap_realm 0x1 0x7f9dc400a590, nref now is 1 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 15 client.4677 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 15 client.4677 snapc 1=[] 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.782 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 put_snap_realm 0x1 0x7f9dc400a590 2 -> 1 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 hrm is_target=1 is_dentry=0 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a680) - ctime 2024-03-28T01:08:18.750146+0000 mtime 2024-03-28T01:08:18.750146+0000 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 dir hash is 2 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 12 client.4677 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 13 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 12 client.4677 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=- has_dir_layout 0x7f9dc400a680) caps pAsLsXsFs 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 get_snap_realm 0x1 0x7f9dc400a590, nref was 1 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 get_snap_realm 0x1 0x7f9dc400a590, nref now is 2 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 15 client.4677 add_update_cap first one, opened snaprealm 0x7f9dc400a590 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 put_snap_realm 0x1 0x7f9dc400a590 2 -> 1 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:18.783 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 lat 0.000691 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 _ll_get 0x7f9dc400a680 0x1 -> 1 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 3 client.4677 (fh)0 is invalid 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 3 client.4677 (fh)0 is invalid 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 2 client.4677 unmount 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 unmounting: trim pass, size was 0+1 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 20 client.4677 trim_cache size 0 max 16384 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9dfde00640 10 client.4677 unmounting: trim pass, size still 0+1 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 2 client.4677 unmounting 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.773+0000 7f9e0324a480 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.784 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 _ll_drop_pins 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 _ll_put 0x7f9dc400a680 0x1 1 -> 0 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 nothing to flush 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 1 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 flush_caps_sync 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 trim_cache size 0 max 16384 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) n = 8 2024-03-28T01:08:18.785 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a680) 2024-03-28T01:08:18.786 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 15 client.4677 remove_cap last one, closing snaprealm 0x7f9dc400a590 2024-03-28T01:08:18.786 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 20 client.4677 put_snap_realm 0x1 0x7f9dc400a590 1 -> 0 2024-03-28T01:08:18.786 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 10 client.4677 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=- has_dir_layout 0x7f9dc400a680) 2024-03-28T01:08:18.786 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 2 client.4677 _close_mds_session mds.0 seq 0 2024-03-28T01:08:18.786 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83faadc0 con 0x560b83fa9590 2024-03-28T01:08:18.786 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.774+0000 7f9e0324a480 2 client.4677 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:18.789 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b83fa9590 2024-03-28T01:08:18.789 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 10 client.4677 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 5 client.4677 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 1 -- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fa9590 msgr2=0x560b83fb44b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fa9590 0x560b83fb44b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 10 client.4677 remove_session_caps mds.0 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 10 client.4677 kick_requests_closed for mds.0 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 10 client.4677 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 20 client.4677 trim_cache size 0 max 16384 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9dfde00640 10 client.4677 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 2 client.4677 unmounted. 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 client.4677 shutdown 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f2c6a0 msgr2=0x560b84059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f2c6a0 0x560b84059ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x560b8404f5e0 tx=0x560b8406b230 comp rx=0 tx=0).stop 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 shutdown_connections 2024-03-28T01:08:18.790 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83fa9590 0x560b83fb44b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f2c6a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 --2- 172.21.15.33:0/967273025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 >> 172.21.15.33:0/967273025 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 shutdown_connections 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 1 -- 172.21.15.33:0/967273025 wait complete. 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.778+0000 7f9e0324a480 20 client.4677 trim_cache size 0 max 16384 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.779+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.779+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83fb44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4007070 con 0x560b83fad220 2024-03-28T01:08:18.791 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83fa9290 con 0x560b83fa9590 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b840612b0 con 0x560b83fb16a0 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49530/0 (socket says 172.21.15.33:49530) 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9dff200640 1 -- 172.21.15.33:0/1992140736 learned_addr learned my addr 172.21.15.33:0/1992140736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.780+0000 7f9e00600640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83fb44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.781+0000 7f9dff200640 1 -- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 msgr2=0x560b83fb44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.781+0000 7f9dff200640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83fb44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.781+0000 7f9dff200640 1 -- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.781+0000 7f9dff200640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.781+0000 7f9dff200640 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b840568a0 con 0x560b83fad220 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.781+0000 7f9dff200640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b84059ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x560b83fbb270 tx=0x7f9de8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9917cf2d6fe8943b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.781+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8068070 con 0x560b83fad220 2024-03-28T01:08:18.792 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 10 client.4680 Subscribing to map 'mdsmap' 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de8050b70 con 0x560b83fad220 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83f2b150 con 0x560b83fad220 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9df6000640 20 client.4680 tick 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9df6000640 20 client.4680 collect_and_send_metrics 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9df6000640 20 client.4680 collect_and_send_global_metrics 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9df6000640 5 client.4680 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9df6000640 20 client.4680 trim_cache size 0 max 16384 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9df6000640 20 client.4680 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de4009f30 con 0x560b83fad220 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025040 con 0x560b83fad220 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9dfca00640 1 client.4680 handle_mds_map epoch 8 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 20 client.4680 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 10 client.4680 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.793 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 20 client.4680 mds is 0 2024-03-28T01:08:18.794 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 10 client.4680 _open_mds_session mds.0 2024-03-28T01:08:18.794 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.794 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:18.794 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9e0324a480 10 client.4680 waiting for session to mds.0 to open 2024-03-28T01:08:18.794 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.782+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.794 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.783+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb8730 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8938cf1fbc9f22f server_cookie=7cc17e4c9391b179 in_seq=0 out_seq=0 2024-03-28T01:08:18.794 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.783+0000 7f9dfca00640 10 client.4680 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:18.796 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.787+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:18.796 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.787+0000 7f9dfca00640 10 client.4680 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:18.796 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.787+0000 7f9dfca00640 10 client.4680 renew_caps mds.0 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a270 con 0x560b84056300 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 connect_mds_targets for mds.0 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 10 client.4680 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 mds is 0 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 10 client.4680 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 encode_cap_releases enter (req: 0x560b83faa6a0, mds: 0) 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 send_request set sent_stamp to 2024-03-28T01:08:18.788746+0000 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 10 client.4680 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.783322+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.783322+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83ee44a0 con 0x560b84056300 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 awaiting reply|forward|kick on 0x7ffe68269800 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a270 con 0x560b84056300 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0019c40 con 0x560b84056300 2024-03-28T01:08:18.797 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 insert_trace from 2024-03-28T01:08:18.788746+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 features 0xffffffffffffffff 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 update_snap_trace len 70 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref now is 1 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 15 client.4680 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 15 client.4680 snapc 1=[] 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 put_snap_realm 0x1 0x7f9dc400a490 2 -> 1 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 hrm is_target=1 is_dentry=0 2024-03-28T01:08:18.798 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a580) - ctime 2024-03-28T01:08:18.750146+0000 mtime 2024-03-28T01:08:18.750146+0000 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 dir hash is 2 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 12 client.4680 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 13 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 12 client.4680 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=- has_dir_layout 0x7f9dc400a580) caps pAsLsXsFs 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref was 1 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref now is 2 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 15 client.4680 add_update_cap first one, opened snaprealm 0x7f9dc400a490 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 10 client.4680 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 put_snap_realm 0x1 0x7f9dc400a490 2 -> 1 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 handle_client_reply signalling caller 0x7ffe68269800 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 lat 0.000769 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 _ll_get 0x7f9dc400a580 0x1 -> 1 2024-03-28T01:08:18.799 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9dfca00640 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 8 client.4680 _ll_create 0x1.head test_llreadvllwritevinvalidfhfile44790 0666 578, uid 0, gid 0 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 10 client.4680 _do_lookup on #0x1/test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 choose_target_mds inode dir hash is 2 on test_llreadvllwritevinvalidfhfile44790 => 918435274 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 choose_target_mds 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) is_hash=1 hash=918435274 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 10 client.4680 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 mds is 0 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 10 client.4680 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 encode_cap_releases enter (req: 0x560b83ee44a0, mds: 0) 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 send_request set sent_stamp to 2024-03-28T01:08:18.789601+0000 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 10 client.4680 send_request client_request(unknown.0:2 lookup #0x1/test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.789565+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.789565+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84053140 con 0x560b84056300 2024-03-28T01:08:18.800 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.788+0000 7f9e0324a480 20 client.4680 awaiting reply|forward|kick on 0x7ffe682695b0 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df004adc0 con 0x560b84056300 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9dfca00640 20 client.4680 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9dfca00640 10 client.4680 insert_trace from 2024-03-28T01:08:18.789601+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9dfca00640 10 client.4680 insert_trace -- no trace 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9dfca00640 20 client.4680 handle_client_reply signalling caller 0x7ffe682695b0 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9dfca00640 20 client.4680 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 lat 0.000582 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 10 client.4680 _do_lookup res is -2 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 10 client.4680 _lookup 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) test_llreadvllwritevinvalidfhfile44790 = -2 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 may_create 0x1.head(faked_ino=0 nref=7 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 10 client.4680 _getattr mask As issued=1 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 3 client.4680 may_create 0x7f9dc400a580 = 0 2024-03-28T01:08:18.801 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 8 client.4680 _create(0x1 test_llreadvllwritevinvalidfhfile44790, 0666) 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 get_or_create 0x1.head(faked_ino=0 nref=8 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) name test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 15 open_dir 0x560b83fb4a60 on 0x7f9dc400a580 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 15 client.4680 link dir 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 'test_llreadvllwritevinvalidfhfile44790' dn 0x1.head["test_llreadvllwritevinvalidfhfile44790"] (new dn) 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 choose_target_mds inode dir hash is 2 on test_llreadvllwritevinvalidfhfile44790 => 918435274 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 choose_target_mds 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) is_hash=1 hash=918435274 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 10 client.4680 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 mds is 0 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 10 client.4680 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 encode_cap_releases enter (req: 0x560b84053140, mds: 0) 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 encode_dentry_release enter(dn:0x7f9dc4009e90) 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580), req:0x560b84053140 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 send_request set sent_stamp to 2024-03-28T01:08:18.790287+0000 2024-03-28T01:08:18.802 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 10 client.4680 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.790240+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.803 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.790240+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84053530 con 0x560b84056300 2024-03-28T01:08:18.803 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9e0324a480 20 client.4680 awaiting reply|forward|kick on 0x7ffe68269640 2024-03-28T01:08:18.803 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.789+0000 7f9dfca00640 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.807 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 859+0+0 (crc 0 0 0) 0x7f9df00548c0 con 0x560b84056300 2024-03-28T01:08:18.807 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 insert_trace from 2024-03-28T01:08:18.790287+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 features 0xffffffffffffffff 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 update_snap_trace len 70 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref was 1 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref now is 2 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 hrm is_target=1 is_dentry=1 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 update_inode_file_time 0x10000000007.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) - ctime 2024-03-28T01:08:18.790240+0000 mtime 2024-03-28T01:08:18.790240+0000 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 12 client.4680 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 12 client.4680 add_update_inode adding 0x10000000007.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=- objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref was 2 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref now is 3 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 15 client.4680 add_update_cap first one, opened snaprealm 0x7f9dc400a490 2024-03-28T01:08:18.808 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000007.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100666 size=0/0 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 12 client.4680 add_update_inode had 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) caps pAsLsXsFs 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 update_inode_file_time 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.750146+0000 ctime=2024-03-28T01:08:18.750146+0000 change_attr=13 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) pAsLsXs ctime 2024-03-28T01:08:18.790240+0000 mtime 2024-03-28T01:08:18.790240+0000 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 dir hash is 2 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 12 client.4680 add_update_inode client inode change_attr: 13 , mds inodestat change_attr: 14 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 10 client.4680 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 12 client.4680 insert_dentry_inode 'test_llreadvllwritevinvalidfhfile44790' vino 0x10000000007.head in dir 0x1.head dn 0x7f9dc4009e90 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 15 client.4680 link dir 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 'test_llreadvllwritevinvalidfhfile44790' to inode 0x7f9dc400ab60 dn 0x1.head["test_llreadvllwritevinvalidfhfile44790"] (old dn) 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 link inode 0x7f9dc400ab60 parents now 0x1.head["test_llreadvllwritevinvalidfhfile44790"] 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.809 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 15 client.4680 update_dentry_lease 0x1.head["test_llreadvllwritevinvalidfhfile44790"] lease(mask 0 dur 0) from 2024-03-28T01:08:18.790287+0000 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 put_snap_realm 0x1 0x7f9dc400a490 3 -> 2 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 handle_client_reply signalling caller 0x7ffe68269640 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9dfca00640 20 client.4680 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9e0324a480 20 client.4680 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9e0324a480 10 client.4680 delegated_inos: [] 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9e0324a480 10 client.4680 make_request created ino 0x10000000007 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9e0324a480 20 client.4680 make_request target is 0x10000000007.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9e0324a480 20 client.4680 lat 0.006367 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.795+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000007.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000007.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 _create_fh 0x10000000007 mode 3 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 trim_cache size 1 max 16384 2024-03-28T01:08:18.810 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 8 client.4680 create(#0x1/test_llreadvllwritevinvalidfhfile44790, 0100666 layout 0 0 0) = 0 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 _ll_create created = 1 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 8 client.4680 _ll_create 0x1.head test_llreadvllwritevinvalidfhfile44790 0666 578 = 0 (0x560b84053530 10000000007) 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 _ll_get 0x7f9dc400ab60 0x10000000007 -> 1 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 fill_statx on 0x10000000007 snap/devhead mode 0100666 mtime 2024-03-28T01:08:18.790240+0000 ctime 2024-03-28T01:08:18.790240+0000 change_attr 0 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 3 client.4680 ll_release (fh)0x560b84053530 0x10000000007 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 8 client.4680 _release_fh 0x560b84053530 mode 3 on 0x10000000007.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={3=1} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 _flush 0x10000000007.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 nothing to flush 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 check_caps on 0x10000000007.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) wanted - used - issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 delaying cap release 2024-03-28T01:08:18.811 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 cap_delay_requeue on 0x10000000007.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 _release_fh 0x560b84053530 on inode 0x10000000007.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) no async_err state 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 3 client.4680 ll_unlink 0x1.head test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 may_delete 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580); ; name test_llreadvllwritevinvalidfhfile44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 _getattr mask As issued=1 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 3 client.4680 may_delete 0x7f9dc400a580 = 0 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 8 client.4680 _unlink(0x1 test_llreadvllwritevinvalidfhfile44790 uid 0 gid 0) 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 get_or_create 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) name test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 _lookup have 0x1.head["test_llreadvllwritevinvalidfhfile44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 _lookup 0x1.head(faked_ino=0 nref=9 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) test_llreadvllwritevinvalidfhfile44790 = 0x10000000007.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000007.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.812 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000007.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 choose_target_mds inode dir hash is 2 on test_llreadvllwritevinvalidfhfile44790 => 918435274 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 choose_target_mds 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580) is_hash=1 hash=918435274 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 choose_target_mds from dirfragtree hash 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 mds is 0 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 encode_cap_releases enter (req: 0x560b84053a70, mds: 0) 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 encode_inode_release enter(in:0x10000000007.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60), req:0x560b84053a70 mds:0, drop:LsxXx, unless:-, force:0) 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 encode_dentry_release enter(dn:0x7f9dc4009e90) 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a580), req:0x560b84053a70 mds:0, drop:Fs, unless:Fx, force:1) 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 send_request set sent_stamp to 2024-03-28T01:08:18.796921+0000 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 10 client.4680 send_request client_request(unknown.0:4 unlink #0x1/test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.796876+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 unlink #0x1/test_llreadvllwritevinvalidfhfile44790 2024-03-28T01:08:18.796876+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84053dd0 con 0x560b84056300 2024-03-28T01:08:18.813 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9e0324a480 20 client.4680 awaiting reply|forward|kick on 0x7ffe68269820 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(revoke ino 0x10000000007 1 seq 2 caps=pAsXsFsxcrwb dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.790240+0000 ctime 2024-03-28T01:08:18.790240+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0094040 con 0x560b84056300 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 5 client.4680 set_cap_epoch_barrier epoch = 20 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 mds.0 seq now 1 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 5 client.4680 handle_cap_grant on in 0x10000000007 mds.0 seq 2 caps now pAsXsFsxcrwb was pAsxXsFsxcrwb 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 update_inode_file_time 0x10000000007.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsxXsFsxcrwb(0=pAsxXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) pAsxXsFsxcrwb ctime 2024-03-28T01:08:18.790240+0000 mtime 2024-03-28T01:08:18.790240+0000 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 revocation of Ax 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 check_caps on 0x10000000007.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) wanted - used - issued pAsXsFsxcrwb revoking Ax flags=1 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsFsxcrwb revoking Ax 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 completed revocation of Ax 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 send_cap 0x10000000007.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["test_llreadvllwritevinvalidfhfile44790"] 0x7f9dc400ab60) mds.0 seq 2 used - want - flush - retain pAsLsXsFs held pAsxXsFsxcrwb revoking Ax dropping Fxcrwb 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 15 client.4680 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:18.814 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000007 1 seq 2 caps=pAsXsFs dirty=- wanted=- follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.790240+0000 ctime 2024-03-28T01:08:18.790240+0000 change_attr 0) v12 -- 0x7f9dc400b140 con 0x560b84056300 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.796+0000 7f9dfca00640 10 client.4680 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:4 = 0 (0) Success unsafe) v1 ==== 507+0+0 (crc 0 0 0) 0x7f9df0063700 con 0x560b84056300 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 handle_client_reply got a reply. Safe:0 tid 4 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 insert_trace from 2024-03-28T01:08:18.796921+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 features 0xffffffffffffffff 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 update_snap_trace len 70 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref was 2 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 get_snap_realm 0x1 0x7f9dc400a490, nref now is 3 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 hrm is_target=0 is_dentry=1 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 12 client.4680 add_update_inode had 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) caps pAsLsXs 2024-03-28T01:08:18.815 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 update_inode_file_time 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=14 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) pAsLsXs ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 dir hash is 2 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 12 client.4680 add_update_inode client inode change_attr: 14 , mds inodestat change_attr: 15 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 15 client.4680 unlink dir 0x7f9dc400a580 'test_llreadvllwritevinvalidfhfile44790' dn 0x7f9dc4009e90 inode 0x7f9dc400ab60 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 unlink inode 0x7f9dc400ab60 parents now 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 put_snap_realm 0x1 0x7f9dc400a490 3 -> 2 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 handle_client_reply signalling caller 0x7ffe68269820 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 handle_client_reply awaiting kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 20 client.4680 sendrecv kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 20 client.4680 lat 0.001431 2024-03-28T01:08:18.816 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 20 client.4680 trim_cache size 1 max 16384 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 8 client.4680 unlink(#0x1/test_llreadvllwritevinvalidfhfile44790) = 0 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=1 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 3 client.4680 (fh)0x560b84053530 is invalid 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 3 client.4680 (fh)0x560b84053530 is invalid 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 2 client.4680 unmount 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, size was 1+2 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 20 client.4680 trim_cache size 1 max 16384 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, size still 1+2 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 2 client.4680 unmounting 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b84053dd0 con 0x560b84056300 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.797+0000 7f9e0324a480 10 client.4680 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_caps(grant ino 0x10000000007 1 seq 3 caps=pAsLsXsFsxcrwb dirty=- wanted=Ls follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.790240+0000 ctime 2024-03-28T01:08:18.796876+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400b140 con 0x560b84056300 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 mds.0 seq now 2 2024-03-28T01:08:18.817 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 5 client.4680 handle_cap_grant on in 0x10000000007 mds.0 seq 3 caps now pAsLsXsFsxcrwb was pAsXsFs 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 update_inode_file_time 0x10000000007.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/4194304 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.790240+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) pAsXsFs ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.790240+0000 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 max_size 4194304 -> 0 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 grant, new caps are LsFxcrwb 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 5 client.4680 _try_to_trim_inode in 0x10000000007.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, size was 1+2 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 20 client.4680 trim_cache size 1 max 16384 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, size still 1+2 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:4 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df00638f0 con 0x560b84056300 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 20 client.4680 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 insert_trace from 2024-03-28T01:08:18.796921+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:18.818 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, size was 1+2 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 20 client.4680 trim_cache size 1 max 16384 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 15 client.4680 trim_dentry unlinking dn test_llreadvllwritevinvalidfhfile44790 in dir 0x1 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 15 client.4680 unlink dir 0x7f9dc400a580 'test_llreadvllwritevinvalidfhfile44790' dn 0x7f9dc4009e90 inode 0 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 15 client.4680 unlink removing 'test_llreadvllwritevinvalidfhfile44790' dn 0x7f9dc4009e90 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 15 client.4680 close_dir dir 0x560b83fb4a60 on 0x7f9dc400a580 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=2 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _ll_drop_pins 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 _ll_put 0x7f9dc400ab60 0x10000000007 1 -> 0 2024-03-28T01:08:18.819 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 _ll_put 0x7f9dc400a580 0x1 2 -> 0 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 _release 0x10000000007.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _invalidate_inode_cache 0x10000000007.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _flush 0x10000000007.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 nothing to flush 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 _release 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _flush 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 nothing to flush 2024-03-28T01:08:18.820 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x10000000007.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 1 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 1 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 flush_caps_sync 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 trim_cache size 0 max 16384 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) n = 12 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 15 client.4680 remove_cap last one, closing snaprealm 0x7f9dc400a490 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_snap_realm 0x1 0x7f9dc400a490 2 -> 1 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a580) 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _put_inode on 0x10000000007.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) n = 15 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 remove_cap mds.0 on 0x10000000007.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) 2024-03-28T01:08:18.821 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 15 client.4680 remove_cap last one, closing snaprealm 0x7f9dc400a490 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 20 client.4680 put_snap_realm 0x1 0x7f9dc400a490 1 -> 0 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 10 client.4680 _put_inode deleting 0x10000000007.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={3=0} mode=100666 size=0/0 nlink=0 btime=2024-03-28T01:08:18.790240+0000 mtime=2024-03-28T01:08:18.790240+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=1 caps=- objectset[0x10000000007 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400ab60) 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 2 client.4680 _close_mds_session mds.0 seq 2 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 2) v7 -- 0x560b83fa6430 con 0x560b84056300 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9e0324a480 2 client.4680 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(grant ino 0x10000000007 1 seq 4 caps=pAsLsXsFsxcrwb dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:18.790240+0000 ctime 2024-03-28T01:08:18.796876+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0063c50 con 0x560b84056300 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 mds.0 seq now 3 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 3) v7 -- 0x7f9dc400a270 con 0x560b84056300 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 5 client.4680 handle_caps don't have vino 0x10000000007.head op 0, immediately releasing 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_cap_release(3) v2 -- 0x560b84053690 con 0x560b84056300 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 20 client.4680 trim_cache size 0 max 16384 2024-03-28T01:08:18.822 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.803+0000 7f9dfca00640 10 client.4680 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.807+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b84053dd0 con 0x560b84056300 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.807+0000 7f9dfca00640 10 client.4680 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.807+0000 7f9dfca00640 5 client.4680 _closed_mds_session mds.0 seq 3 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.807+0000 7f9dfca00640 1 -- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83fb8730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9dfca00640 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb8730 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9dfca00640 10 client.4680 remove_session_caps mds.0 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9dfca00640 10 client.4680 kick_requests_closed for mds.0 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 2 client.4680 unmounted. 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 client.4680 shutdown 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 msgr2=0x560b84059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b84059ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x560b83fbb270 tx=0x7f9de8012000 comp rx=0 tx=0).stop 2024-03-28T01:08:18.824 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 shutdown_connections 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb8730 unknown :-1 s=CLOSED pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83fb44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1992140736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 >> 172.21.15.33:0/1992140736 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 shutdown_connections 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 1 -- 172.21.15.33:0/1992140736 wait complete. 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.808+0000 7f9e0324a480 20 client.4680 trim_cache size 0 max 16384 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.LlreadvLlwritevInvalidFileHandle (45 ms) 2024-03-28T01:08:18.825 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.CheckDummyOP 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.809+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4004ba0 con 0x560b83fa9590 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2dd50 con 0x560b83fb16a0 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fb1a80 con 0x560b83fad220 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.810+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49532/0 (socket says 172.21.15.33:49532) 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dffc00640 1 -- 172.21.15.33:0/894028930 learned_addr learned my addr 172.21.15.33:0/894028930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9e00600640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.832 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dff200640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dffc00640 1 -- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 msgr2=0x560b83fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dffc00640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dffc00640 1 -- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83f2cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dffc00640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dffc00640 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f27ea0 con 0x560b83fa9590 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dffc00640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b84059ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9df001bd70 tx=0x7f9df008e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57057b8af5cf370a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.811+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00ae060 con 0x560b83fa9590 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 10 client.4683 Subscribing to map 'mdsmap' 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9df00634a0 con 0x560b83fa9590 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83fbb540 con 0x560b83fa9590 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9df5600640 20 client.4683 tick 2024-03-28T01:08:18.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9df5600640 20 client.4683 collect_and_send_metrics 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9df5600640 20 client.4683 collect_and_send_global_metrics 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9df5600640 5 client.4683 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9df5600640 20 client.4683 trim_cache size 0 max 16384 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9df5600640 20 client.4683 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b84053690 con 0x560b83fa9590 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0091040 con 0x560b83fa9590 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9dfde00640 1 client.4683 handle_mds_map epoch 8 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 20 client.4683 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 10 client.4683 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 20 client.4683 mds is 0 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 10 client.4683 _open_mds_session mds.0 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:18.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9e0324a480 10 client.4683 waiting for session to mds.0 to open 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.812+0000 7f9dff200640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.813+0000 7f9dff200640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fae340 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41816aee294e182 server_cookie=f9fa1f324047aea6 in_seq=0 out_seq=0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.813+0000 7f9dfde00640 10 client.4683 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 renew_caps mds.0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc40088c0 con 0x560b84056300 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 connect_mds_targets for mds.0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 10 client.4683 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 mds is 0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 10 client.4683 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 encode_cap_releases enter (req: 0x560b83f2b100, mds: 0) 2024-03-28T01:08:18.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 send_request set sent_stamp to 2024-03-28T01:08:18.817969+0000 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 10 client.4683 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.813252+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.813252+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84052930 con 0x560b84056300 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc40088c0 con 0x560b84056300 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9dc40030b0 con 0x560b84056300 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 insert_trace from 2024-03-28T01:08:18.817969+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 features 0xffffffffffffffff 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 update_snap_trace len 70 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 get_snap_realm 0x1 0x7f9dc400a580, nref now is 1 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:18.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 15 client.4683 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 15 client.4683 snapc 1=[] 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 put_snap_realm 0x1 0x7f9dc400a580 2 -> 1 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 hrm is_target=1 is_dentry=0 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a670) - ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 dir hash is 2 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 12 client.4683 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 15 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 12 client.4683 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) caps pAsLsXsFs 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 get_snap_realm 0x1 0x7f9dc400a580, nref was 1 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 get_snap_realm 0x1 0x7f9dc400a580, nref now is 2 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 15 client.4683 add_update_cap first one, opened snaprealm 0x7f9dc400a580 2024-03-28T01:08:18.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 10 client.4683 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 put_snap_realm 0x1 0x7f9dc400a580 2 -> 1 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 lat 0.000507 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 _ll_get 0x7f9dc400a670 0x1 -> 1 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 10 client.4683 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 mds is 0 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 10 client.4683 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 1 client.4683 Warning -- unable to construct a filepath! No path, inode, or dentry given! 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 encode_cap_releases enter (req: 0x560b84052930, mds: 0) 2024-03-28T01:08:18.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 send_request set sent_stamp to 2024-03-28T01:08:18.818493+0000 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 10 client.4683 send_request client_request(unknown.0:2 dummy 2024-03-28T01:08:18.818485+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 dummy 2024-03-28T01:08:18.818485+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83faa5e0 con 0x560b84056300 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9e0324a480 20 client.4683 awaiting reply|forward|kick on 0x7ffe68269c80 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.817+0000 7f9dfde00640 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -95 (95) Operation not supported) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de80109f0 con 0x560b84056300 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 20 client.4683 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 10 client.4683 insert_trace from 2024-03-28T01:08:18.818493+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 10 client.4683 insert_trace -- no trace 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 20 client.4683 handle_client_reply signalling caller 0x7ffe68269c80 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 20 client.4683 handle_client_reply awaiting kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 sendrecv kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:18.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 lat 0.000282 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 check_dummy_op result=-95 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 2 client.4683 unmount 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 2 client.4683 unmounting 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83faa5e0 con 0x560b84056300 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 _ll_drop_pins 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 _ll_put 0x7f9dc400a670 0x1 1 -> 0 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 nothing to flush 2024-03-28T01:08:18.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 flush_caps_sync 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 trim_cache size 0 max 16384 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 8 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 15 client.4683 remove_cap last one, closing snaprealm 0x7f9dc400a580 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 20 client.4683 put_snap_realm 0x1 0x7f9dc400a580 1 -> 0 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 10 client.4683 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 2 client.4683 _close_mds_session mds.0 seq 0 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83f28d60 con 0x560b84056300 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9e0324a480 2 client.4683 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:18.841 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 10 client.4683 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 20 client.4683 trim_cache size 0 max 16384 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.818+0000 7f9dfde00640 10 client.4683 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83f28d60 con 0x560b84056300 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 10 client.4683 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 5 client.4683 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 1 -- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83fae340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fae340 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 10 client.4683 remove_session_caps mds.0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 10 client.4683 kick_requests_closed for mds.0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 10 client.4683 unmounting: trim pass, size was 0+0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 20 client.4683 trim_cache size 0 max 16384 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9dfde00640 10 client.4683 unmounting: trim pass, size still 0+0 2024-03-28T01:08:18.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9e0324a480 2 client.4683 unmounted. 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.821+0000 7f9e0324a480 1 client.4683 shutdown 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 msgr2=0x560b84059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b84059ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9df001bd70 tx=0x7f9df008e000 comp rx=0 tx=0).stop 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 shutdown_connections 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fae340 unknown :-1 s=CLOSED pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 --2- 172.21.15.33:0/894028930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b84059ee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 >> 172.21.15.33:0/894028930 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 shutdown_connections 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 1 -- 172.21.15.33:0/894028930 wait complete. 2024-03-28T01:08:18.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.822+0000 7f9e0324a480 20 client.4683 trim_cache size 0 max 16384 2024-03-28T01:08:18.844 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.CheckDummyOP (14 ms) 2024-03-28T01:08:18.844 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.CheckUnknownSessionOp 2024-03-28T01:08:18.847 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.823+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:18.847 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.823+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:18.847 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.847 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.847 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4003ef0 con 0x560b83fad220 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f27890 con 0x560b83fa9590 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b840612b0 con 0x560b83fb16a0 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49544/0 (socket says 172.21.15.33:49544) 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9dff200640 1 -- 172.21.15.33:0/4184856115 learned_addr learned my addr 172.21.15.33:0/4184856115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.824+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9e00600640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dff200640 1 -- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dff200640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dff200640 1 -- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.848 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dff200640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dff200640 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83ee44a0 con 0x560b83fad220 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dff200640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9de8010960 tx=0x7f9de806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c713bb92a8790b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8065070 con 0x560b83fad220 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f9de8073020 con 0x560b83fad220 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9e0324a480 10 client.4686 Subscribing to map 'mdsmap' 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.825+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b84056a70 con 0x560b83fad220 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9df6000640 20 client.4686 tick 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9df6000640 20 client.4686 collect_and_send_metrics 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9df6000640 20 client.4686 collect_and_send_global_metrics 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9df6000640 5 client.4686 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9df6000640 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9df6000640 20 client.4686 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:18.849 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de80109f0 con 0x560b83fad220 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025040 con 0x560b83fad220 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9dfca00640 1 client.4686 handle_mds_map epoch 8 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9e0324a480 20 client.4686 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9e0324a480 10 client.4686 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9e0324a480 20 client.4686 mds is 0 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9e0324a480 10 client.4686 _open_mds_session mds.0 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9e0324a480 10 client.4686 waiting for session to mds.0 to open 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.826+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.827+0000 7f9dffc00640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bda7429b6992b7e server_cookie=dc3adb214728ee93 in_seq=0 out_seq=0 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.827+0000 7f9dfca00640 10 client.4686 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:18.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9dfca00640 10 client.4686 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9dfca00640 10 client.4686 renew_caps mds.0 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a1a0 con 0x560b84056300 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9dfca00640 10 client.4686 connect_mds_targets for mds.0 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 10 client.4686 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 20 client.4686 mds is 0 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 10 client.4686 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 20 client.4686 encode_cap_releases enter (req: 0x560b83f2e280, mds: 0) 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 20 client.4686 send_request set sent_stamp to 2024-03-28T01:08:18.831327+0000 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 10 client.4686 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.827062+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:18.827062+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fae340 con 0x560b84056300 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9e0324a480 20 client.4686 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a1a0 con 0x560b84056300 2024-03-28T01:08:18.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.830+0000 7f9dfca00640 10 client.4686 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df001ebf0 con 0x560b84056300 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 insert_trace from 2024-03-28T01:08:18.831327+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 features 0xffffffffffffffff 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 update_snap_trace len 70 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 get_snap_realm 0x1 0x7f9dc400a580, nref now is 1 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 15 client.4686 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 15 client.4686 snapc 1=[] 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 put_snap_realm 0x1 0x7f9dc400a580 2 -> 1 2024-03-28T01:08:18.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 hrm is_target=1 is_dentry=0 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a670) - ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 dir hash is 2 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 12 client.4686 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 15 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 12 client.4686 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) caps pAsLsXsFs 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 get_snap_realm 0x1 0x7f9dc400a580, nref was 1 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 get_snap_realm 0x1 0x7f9dc400a580, nref now is 2 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 15 client.4686 add_update_cap first one, opened snaprealm 0x7f9dc400a580 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 add_update_cap calling signal_caps_inode 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 put_snap_realm 0x1 0x7f9dc400a580 2 -> 1 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9e0324a480 20 client.4686 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:18.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9e0324a480 20 client.4686 lat 0.000675 2024-03-28T01:08:18.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9e0324a480 20 client.4686 _ll_get 0x7f9dc400a670 0x1 -> 1 2024-03-28T01:08:18.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(???) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:18.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9e0324a480 10 send_unknown_session_op result=0 2024-03-28T01:08:18.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.854 INFO:tasks.ceph.mds.a.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f0942000640 -1 mds.0.server Server received unknown message 22, closing session and blocklisting the client 4686 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_session(reject) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 handle_client_session client_session(reject) v1 from mds.0 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 -1 client.4686 mds.0 rejected us (unknown error) 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 5 client.4686 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83f29260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 remove_session_caps mds.0 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=7 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 15 client.4686 remove_cap last one, closing snaprealm 0x7f9dc400a580 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 put_snap_realm 0x1 0x7f9dc400a580 1 -> 0 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 remove_session_caps calling signal_caps_inode 2024-03-28T01:08:18.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:18.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:18.831+0000 7f9dfca00640 10 client.4686 kick_requests_closed for mds.0 2024-03-28T01:08:19.828 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 20 client.4686 tick 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 10 client.4686 renew_caps() 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 15 client.4686 renew_caps requesting from mds.0 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 10 client.4686 renew_caps mds.0 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 2) v7 -- 0x7f9dcc0099b0 con 0x560b84056300 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 20 client.4686 collect_and_send_metrics 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 20 client.4686 collect_and_send_global_metrics 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 5 client.4686 collect_and_send_global_metrics: no session with rank=0 -- not sending metric 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 10 client.4686 _put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 3 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:19.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:19.826+0000 7f9df6000640 20 client.4686 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:20.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:20.826+0000 7f9df6000640 20 client.4686 tick 2024-03-28T01:08:20.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:20.826+0000 7f9df6000640 20 client.4686 collect_and_send_metrics 2024-03-28T01:08:20.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:20.826+0000 7f9df6000640 20 client.4686 collect_and_send_global_metrics 2024-03-28T01:08:20.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:20.826+0000 7f9df6000640 5 client.4686 collect_and_send_global_metrics: no session with rank=0 -- not sending metric 2024-03-28T01:08:20.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:20.826+0000 7f9df6000640 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:20.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:20.826+0000 7f9df6000640 20 client.4686 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:21.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:21.826+0000 7f9df6000640 20 client.4686 tick 2024-03-28T01:08:21.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:21.826+0000 7f9df6000640 20 client.4686 collect_and_send_metrics 2024-03-28T01:08:21.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:21.826+0000 7f9df6000640 20 client.4686 collect_and_send_global_metrics 2024-03-28T01:08:21.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:21.826+0000 7f9df6000640 5 client.4686 collect_and_send_global_metrics: no session with rank=0 -- not sending metric 2024-03-28T01:08:21.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:21.826+0000 7f9df6000640 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:21.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:21.826+0000 7f9df6000640 20 client.4686 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:22.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:22.826+0000 7f9df6000640 20 client.4686 tick 2024-03-28T01:08:22.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:22.826+0000 7f9df6000640 20 client.4686 collect_and_send_metrics 2024-03-28T01:08:22.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:22.826+0000 7f9df6000640 20 client.4686 collect_and_send_global_metrics 2024-03-28T01:08:22.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:22.826+0000 7f9df6000640 5 client.4686 collect_and_send_global_metrics: no session with rank=0 -- not sending metric 2024-03-28T01:08:22.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:22.826+0000 7f9df6000640 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:22.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:22.826+0000 7f9df6000640 20 client.4686 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.826+0000 7f9df6000640 20 client.4686 tick 2024-03-28T01:08:23.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.826+0000 7f9df6000640 20 client.4686 collect_and_send_metrics 2024-03-28T01:08:23.829 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.826+0000 7f9df6000640 20 client.4686 collect_and_send_global_metrics 2024-03-28T01:08:23.830 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.826+0000 7f9df6000640 5 client.4686 collect_and_send_global_metrics: no session with rank=0 -- not sending metric 2024-03-28T01:08:23.830 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.826+0000 7f9df6000640 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:23.830 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.826+0000 7f9df6000640 20 client.4686 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.833 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.831+0000 7f9e0324a480 20 check_client_blocklisted: waiting for latest osdmap 2024-03-28T01:08:23.834 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.831+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x560b83fae340 con 0x560b83fad220 2024-03-28T01:08:23.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.832+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mon.0 v2:172.21.15.33:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de807c080 con 0x560b83fad220 2024-03-28T01:08:23.835 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.832+0000 7f9dfe800640 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=23}) v3 -- 0x7f9dd00027c0 con 0x560b83fad220 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 <== mon.0 v2:172.21.15.33:3300/0 6 ==== osd_map(23..23 src has 1..23) v4 ==== 331+0+0 (secure 0 0 0) 0x7f9de80109f0 con 0x560b83fad220 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 check_client_blocklisted: got latest osdmap: system:0 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 2 client.4686 unmount 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 2 client.4686 unmounting 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=4 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=4 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 10 client.4686 _ll_drop_pins 2024-03-28T01:08:23.836 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 client.4686 _ll_put 0x7f9dc400a670 0x1 1 -> 0 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=5 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=5 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 client.4686 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 1 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 10 client.4686 flush_caps_sync 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 10 client.4686 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 10 client.4686 _put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) n = 5 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 10 client.4686 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a670) 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 2 client.4686 unmounted. 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9e0324a480 1 client.4686 shutdown 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9dfca00640 -1 client.4686 I was blocklisted at osd epoch 23 2024-03-28T01:08:23.837 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.833+0000 7f9dfca00640 1 -- 172.21.15.33:0/4184856115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=24}) v3 -- 0x7f9dc4009730 con 0x560b83fad220 2024-03-28T01:08:23.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 msgr2=0x560b83fba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:23.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9de8010960 tx=0x7f9de806a000 comp rx=0 tx=0).stop 2024-03-28T01:08:23.838 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 shutdown_connections 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 unknown :-1 s=CLOSED pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 --2- 172.21.15.33:0/4184856115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.834+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 >> 172.21.15.33:0/4184856115 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.835+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 shutdown_connections 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.835+0000 7f9e0324a480 1 -- 172.21.15.33:0/4184856115 wait complete. 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.835+0000 7f9e0324a480 20 client.4686 trim_cache size 0 max 16384 2024-03-28T01:08:23.839 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.CheckUnknownSessionOp (5013 ms) 2024-03-28T01:08:23.840 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.CheckZeroReclaimFlag 2024-03-28T01:08:23.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.836+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:23.840 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.838+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4007f00 con 0x560b83fad220 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2b100 con 0x560b83fa9590 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f520 con 0x560b83fb16a0 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.842 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60884/0 (socket says 172.21.15.33:60884) 2024-03-28T01:08:23.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9dff200640 1 -- 172.21.15.33:0/3967720980 learned_addr learned my addr 172.21.15.33:0/3967720980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:23.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.839+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.843 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9e00600640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.844 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9dff200640 1 -- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 msgr2=0x560b83f2cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.844 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9dff200640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.844 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9dff200640 1 -- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.844 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9dff200640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9dff200640 1 -- 172.21.15.33:0/3967720980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f31e80 con 0x560b83fad220 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9e00600640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9e0324a480 10 client.4689 Subscribing to map 'mdsmap' 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mdsmap=0+}) v3 -- 0x560b83fa9220 con 0x560b83fad220 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9dff200640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x560b83fad670 tx=0x560b83f2ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6d15685d7f7a3a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.840+0000 7f9dfde00640 10 client.4689 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9df5600640 20 client.4689 tick 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9df5600640 20 client.4689 collect_and_send_metrics 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9df5600640 20 client.4689 collect_and_send_global_metrics 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9df5600640 5 client.4689 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9df5600640 20 client.4689 trim_cache size 0 max 16384 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9df5600640 20 client.4689 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.845 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8065070 con 0x560b83fad220 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9de802bbc0 con 0x560b83fad220 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8025070 con 0x560b83fad220 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9dfde00640 1 client.4689 handle_mds_map epoch 8 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9e0324a480 20 client.4689 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9e0324a480 10 client.4689 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9e0324a480 20 client.4689 mds is 0 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9e0324a480 10 client.4689 _open_mds_session mds.0 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.841+0000 7f9e0324a480 10 client.4689 waiting for session to mds.0 to open 2024-03-28T01:08:23.846 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.842+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.847 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.842+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd465bd3575728b0 server_cookie=b80f4b850813ad6f in_seq=0 out_seq=0 2024-03-28T01:08:23.847 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.842+0000 7f9dfde00640 10 client.4689 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:23.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9dfde00640 10 client.4689 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:23.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9dfde00640 10 client.4689 renew_caps mds.0 2024-03-28T01:08:23.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.850 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9dfde00640 10 client.4689 connect_mds_targets for mds.0 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 10 client.4689 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 20 client.4689 mds is 0 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 10 client.4689 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 20 client.4689 encode_cap_releases enter (req: 0x560b8405a0b0, mds: 0) 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 20 client.4689 send_request set sent_stamp to 2024-03-28T01:08:23.848535+0000 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 10 client.4689 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.842262+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.842262+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fae340 con 0x560b84056300 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.847+0000 7f9e0324a480 20 client.4689 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.851 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:23.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b84056300 2024-03-28T01:08:23.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:23.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 insert_trace from 2024-03-28T01:08:23.848535+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:23.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 features 0xffffffffffffffff 2024-03-28T01:08:23.852 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 update_snap_trace len 70 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 get_snap_realm 0x1 0x7f9dc4004fc0, nref now is 1 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 15 client.4689 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 15 client.4689 snapc 1=[] 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 put_snap_realm 0x1 0x7f9dc4004fc0 2 -> 1 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 hrm is_target=1 is_dentry=0 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a7a0) - ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 dir hash is 2 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 12 client.4689 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 15 2024-03-28T01:08:23.853 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 12 client.4689 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) caps pAsLsXsFs 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 get_snap_realm 0x1 0x7f9dc4004fc0, nref was 1 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 get_snap_realm 0x1 0x7f9dc4004fc0, nref now is 2 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 15 client.4689 add_update_cap first one, opened snaprealm 0x7f9dc4004fc0 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 10 client.4689 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 put_snap_realm 0x1 0x7f9dc4004fc0 2 -> 1 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9e0324a480 20 client.4689 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9e0324a480 20 client.4689 lat 0.001010 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9e0324a480 20 client.4689 _ll_get 0x7f9dc400a7a0 0x1 -> 1 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_reclaim(unknownreclaimflag:aef6 flags 0x0) v1 -- 0x560b83fae340 con 0x560b84056300 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.854 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.848+0000 7f9dfde00640 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reclaim_reply(-22 e 0) v1 ==== 13+0+0 (crc 0 0 0) 0x7f9df00b4080 con 0x560b84056300 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9dfde00640 10 client.4689 handle_client_reclaim_reply client_reclaim_reply(-22 e 0) v1 from mds.0 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 2 client.4689 unmount 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 2 client.4689 unmounting 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 _ll_drop_pins 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 _ll_put 0x7f9dc400a7a0 0x1 1 -> 0 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.855 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 nothing to flush 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 flush_caps_sync 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 trim_cache size 0 max 16384 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 8 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 15 client.4689 remove_cap last one, closing snaprealm 0x7f9dc4004fc0 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 20 client.4689 put_snap_realm 0x1 0x7f9dc4004fc0 1 -> 0 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 10 client.4689 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 2 client.4689 _close_mds_session mds.0 seq 0 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.856 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.849+0000 7f9e0324a480 2 client.4689 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:23.857 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 10 client.4689 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 5 client.4689 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 1 -- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83f29260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 10 client.4689 remove_session_caps mds.0 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 10 client.4689 kick_requests_closed for mds.0 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 10 client.4689 unmounting: trim pass, size was 0+0 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 20 client.4689 trim_cache size 0 max 16384 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9dfde00640 10 client.4689 unmounting: trim pass, size still 0+0 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9e0324a480 2 client.4689 unmounted. 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.853+0000 7f9e0324a480 1 client.4689 shutdown 2024-03-28T01:08:23.858 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 msgr2=0x560b83fba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x560b83fad670 tx=0x560b83f2ddc0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 shutdown_connections 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f29260 unknown :-1 s=CLOSED pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3967720980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 >> 172.21.15.33:0/3967720980 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 shutdown_connections 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 1 -- 172.21.15.33:0/3967720980 wait complete. 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.854+0000 7f9e0324a480 20 client.4689 trim_cache size 0 max 16384 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.CheckZeroReclaimFlag (19 ms) 2024-03-28T01:08:23.859 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.CheckUnknownReclaimFlag 2024-03-28T01:08:23.862 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.855+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:23.862 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.855+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:23.862 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.862 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.862 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.862 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4009a10 con 0x560b83fb16a0 2024-03-28T01:08:23.862 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f520 con 0x560b83fa9590 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83f28c40 con 0x560b83fad220 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60896/0 (socket says 172.21.15.33:60896) 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9e00600640 1 -- 172.21.15.33:0/2541881334 learned_addr learned my addr 172.21.15.33:0/2541881334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.856+0000 7f9dff200640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.857+0000 7f9e00600640 1 -- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 msgr2=0x560b83fba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.857+0000 7f9e00600640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.857+0000 7f9e00600640 1 -- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.857+0000 7f9e00600640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.857+0000 7f9e00600640 1 -- 172.21.15.33:0/2541881334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8406bfc0 con 0x560b83fb16a0 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.857+0000 7f9e00600640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001cc0 tx=0x7f9df801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f698275d242dc74 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.857+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:23.863 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8041040 con 0x560b83fb16a0 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df800d4d0 con 0x560b83fb16a0 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 10 client.4692 Subscribing to map 'mdsmap' 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83eff3c0 con 0x560b83fb16a0 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9df6000640 20 client.4692 tick 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9df6000640 20 client.4692 collect_and_send_metrics 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9df6000640 20 client.4692 collect_and_send_global_metrics 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9df6000640 5 client.4692 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9df6000640 20 client.4692 trim_cache size 0 max 16384 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9df6000640 20 client.4692 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8063020 con 0x560b83fb16a0 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b83fb16a0 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9dfca00640 1 client.4692 handle_mds_map epoch 8 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 20 client.4692 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:23.864 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 10 client.4692 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 20 client.4692 mds is 0 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 10 client.4692 _open_mds_session mds.0 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b84056810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.858+0000 7f9e0324a480 10 client.4692 waiting for session to mds.0 to open 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.859+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b84056810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.859+0000 7f9dfca00640 10 client.4692 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:23.865 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.859+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b84056810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b33806217969610 server_cookie=35b9aa3209c382dc in_seq=0 out_seq=0 2024-03-28T01:08:23.867 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.867 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9dfca00640 10 client.4692 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:23.867 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9dfca00640 10 client.4692 renew_caps mds.0 2024-03-28T01:08:23.867 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.867 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9dfca00640 10 client.4692 connect_mds_targets for mds.0 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 10 client.4692 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 20 client.4692 mds is 0 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 10 client.4692 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 20 client.4692 encode_cap_releases enter (req: 0x560b83ee4560, mds: 0) 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 20 client.4692 send_request set sent_stamp to 2024-03-28T01:08:23.864382+0000 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 10 client.4692 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.859395+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.859395+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2a190 con 0x560b84056300 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9e0324a480 20 client.4692 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.863+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b84056300 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 insert_trace from 2024-03-28T01:08:23.864382+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:23.868 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 features 0xffffffffffffffff 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 update_snap_trace len 70 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 get_snap_realm 0x1 0x7f9dc4005080, nref now is 1 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 15 client.4692 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 15 client.4692 snapc 1=[] 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 put_snap_realm 0x1 0x7f9dc4005080 2 -> 1 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 hrm is_target=1 is_dentry=0 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a7a0) - ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 dir hash is 2 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 12 client.4692 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 15 2024-03-28T01:08:23.869 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 12 client.4692 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) caps pAsLsXsFs 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 get_snap_realm 0x1 0x7f9dc4005080, nref was 1 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 get_snap_realm 0x1 0x7f9dc4005080, nref now is 2 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 15 client.4692 add_update_cap first one, opened snaprealm 0x7f9dc4005080 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 put_snap_realm 0x1 0x7f9dc4005080 2 -> 1 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 lat 0.000619 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 _ll_get 0x7f9dc400a7a0 0x1 -> 1 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_reclaim(unknownreclaimflag:aef6 flags 0x2) v1 -- 0x560b83f2a190 con 0x560b84056300 2024-03-28T01:08:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reclaim_reply(-22 e 0) v1 ==== 13+0+0 (crc 0 0 0) 0x7f9df0099040 con 0x560b84056300 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9dfca00640 10 client.4692 handle_client_reclaim_reply client_reclaim_reply(-22 e 0) v1 from mds.0 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 2 client.4692 unmount 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 2 client.4692 unmounting 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 _ll_drop_pins 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 _ll_put 0x7f9dc400a7a0 0x1 1 -> 0 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 nothing to flush 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 flush_caps_sync 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 trim_cache size 0 max 16384 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 8 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 15 client.4692 remove_cap last one, closing snaprealm 0x7f9dc4005080 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 20 client.4692 put_snap_realm 0x1 0x7f9dc4005080 1 -> 0 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 10 client.4692 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 2 client.4692 _close_mds_session mds.0 seq 0 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.864+0000 7f9e0324a480 2 client.4692 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 10 client.4692 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 5 client.4692 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 1 -- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b84056810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b84056810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 10 client.4692 remove_session_caps mds.0 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 10 client.4692 kick_requests_closed for mds.0 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 10 client.4692 unmounting: trim pass, size was 0+0 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 20 client.4692 trim_cache size 0 max 16384 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9dfca00640 10 client.4692 unmounting: trim pass, size still 0+0 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9e0324a480 2 client.4692 unmounted. 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9e0324a480 1 client.4692 shutdown 2024-03-28T01:08:23.874 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83f2cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.870+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001cc0 tx=0x7f9df801c000 comp rx=0 tx=0).stop 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 shutdown_connections 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b84056810 unknown :-1 s=CLOSED pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2541881334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 >> 172.21.15.33:0/2541881334 conn(0x560b83f2d970 msgr2=0x560b83f29260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 shutdown_connections 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 1 -- 172.21.15.33:0/2541881334 wait complete. 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.871+0000 7f9e0324a480 20 client.4692 trim_cache size 0 max 16384 2024-03-28T01:08:23.875 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.CheckUnknownReclaimFlag (17 ms) 2024-03-28T01:08:23.876 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.CheckNegativeReclaimFlagUnmasked 2024-03-28T01:08:23.878 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.872+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:23.878 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.872+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:23.878 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.878 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.878 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.878 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4007ec0 con 0x560b83fb16a0 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83fb8920 con 0x560b83fad220 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83ee61a0 con 0x560b83fa9590 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60904/0 (socket says 172.21.15.33:60904) 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9e00600640 1 -- 172.21.15.33:0/1771628774 learned_addr learned my addr 172.21.15.33:0/1771628774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9dff200640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.873+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e00600640 1 -- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e00600640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e00600640 1 -- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 msgr2=0x560b83fba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e00600640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.879 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e00600640 1 -- 172.21.15.33:0/1771628774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f31e30 con 0x560b83fb16a0 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e00600640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001e00 tx=0x7f9df8054040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4433bcd4b0969f7b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df804c070 con 0x560b83fb16a0 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df8003070 con 0x560b83fb16a0 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e0324a480 10 client.4695 Subscribing to map 'mdsmap' 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83fae690 con 0x560b83fb16a0 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9df5600640 20 client.4695 tick 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9df5600640 20 client.4695 collect_and_send_metrics 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9df5600640 20 client.4695 collect_and_send_global_metrics 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9df5600640 5 client.4695 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9df5600640 20 client.4695 trim_cache size 0 max 16384 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9df5600640 20 client.4695 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.880 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.874+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df806b020 con 0x560b83fb16a0 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b83fb16a0 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9dfde00640 1 client.4695 handle_mds_map epoch 8 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9e0324a480 20 client.4695 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9e0324a480 10 client.4695 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9e0324a480 20 client.4695 mds is 0 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9e0324a480 10 client.4695 _open_mds_session mds.0 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b840566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9e0324a480 10 client.4695 waiting for session to mds.0 to open 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.875+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b840566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.876+0000 7f9dffc00640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b840566e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59398e3abe6cb02d server_cookie=bbc8e0d993235ad4 in_seq=0 out_seq=0 2024-03-28T01:08:23.881 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.876+0000 7f9dfde00640 10 client.4695 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9dfde00640 10 client.4695 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9dfde00640 10 client.4695 renew_caps mds.0 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9dfde00640 10 client.4695 connect_mds_targets for mds.0 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 10 client.4695 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 20 client.4695 mds is 0 2024-03-28T01:08:23.885 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 10 client.4695 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 20 client.4695 encode_cap_releases enter (req: 0x560b83fa9970, mds: 0) 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 20 client.4695 send_request set sent_stamp to 2024-03-28T01:08:23.882257+0000 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 10 client.4695 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.875784+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.875784+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83ef28d0 con 0x560b84056300 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9e0324a480 20 client.4695 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.881+0000 7f9dfde00640 10 client.4695 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df000e000 con 0x560b84056300 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 insert_trace from 2024-03-28T01:08:23.882257+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 features 0xffffffffffffffff 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 update_snap_trace len 70 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:23.886 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 get_snap_realm 0x1 0x7f9dc4005020, nref now is 1 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 15 client.4695 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 15 client.4695 snapc 1=[] 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 put_snap_realm 0x1 0x7f9dc4005020 2 -> 1 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 hrm is_target=1 is_dentry=0 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a7a0) - ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 dir hash is 2 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 12 client.4695 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 15 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 12 client.4695 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) caps pAsLsXsFs 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 get_snap_realm 0x1 0x7f9dc4005020, nref was 1 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 get_snap_realm 0x1 0x7f9dc4005020, nref now is 2 2024-03-28T01:08:23.887 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 15 client.4695 add_update_cap first one, opened snaprealm 0x7f9dc4005020 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 put_snap_realm 0x1 0x7f9dc4005020 2 -> 1 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 lat 0.000588 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 _ll_get 0x7f9dc400a7a0 0x1 -> 1 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_reclaim(unknownreclaimflag:aef6 flags 0x7fffffff) v1 -- 0x560b83ef28d0 con 0x560b84056300 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reclaim_reply(-22 e 0) v1 ==== 13+0+0 (crc 0 0 0) 0x7f9df0099040 con 0x560b84056300 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9dfde00640 10 client.4695 handle_client_reclaim_reply client_reclaim_reply(-22 e 0) v1 from mds.0 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 2 client.4695 unmount 2024-03-28T01:08:23.888 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 2 client.4695 unmounting 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 _ll_drop_pins 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 _ll_put 0x7f9dc400a7a0 0x1 1 -> 0 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 nothing to flush 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 flush_caps_sync 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.889 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 trim_cache size 0 max 16384 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 8 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 15 client.4695 remove_cap last one, closing snaprealm 0x7f9dc4005020 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 20 client.4695 put_snap_realm 0x1 0x7f9dc4005020 1 -> 0 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 10 client.4695 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 2 client.4695 _close_mds_session mds.0 seq 0 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.890 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.882+0000 7f9e0324a480 2 client.4695 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:23.891 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.891 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 10 client.4695 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:23.891 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 5 client.4695 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:23.891 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 1 -- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b840566e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.891 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b840566e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 10 client.4695 remove_session_caps mds.0 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 10 client.4695 kick_requests_closed for mds.0 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 10 client.4695 unmounting: trim pass, size was 0+0 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 20 client.4695 trim_cache size 0 max 16384 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9dfde00640 10 client.4695 unmounting: trim pass, size still 0+0 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9e0324a480 2 client.4695 unmounted. 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.887+0000 7f9e0324a480 1 client.4695 shutdown 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83f2cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9df8001e00 tx=0x7f9df8054040 comp rx=0 tx=0).stop 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 shutdown_connections 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b840566e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.892 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 --2- 172.21.15.33:0/1771628774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.893 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 >> 172.21.15.33:0/1771628774 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:23.893 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 shutdown_connections 2024-03-28T01:08:23.893 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 1 -- 172.21.15.33:0/1771628774 wait complete. 2024-03-28T01:08:23.893 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.888+0000 7f9e0324a480 20 client.4695 trim_cache size 0 max 16384 2024-03-28T01:08:23.893 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.CheckNegativeReclaimFlagUnmasked (17 ms) 2024-03-28T01:08:23.893 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.CheckNegativeReclaimFlag 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.889+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.889+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4003990 con 0x560b83fad220 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2dd50 con 0x560b83fb16a0 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fae4f0 con 0x560b83fa9590 2024-03-28T01:08:23.896 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60906/0 (socket says 172.21.15.33:60906) 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.890+0000 7f9dff200640 1 -- 172.21.15.33:0/2837783121 learned_addr learned my addr 172.21.15.33:0/2837783121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9e00600640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dff200640 1 -- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dff200640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dff200640 1 -- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83f2cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dff200640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dff200640 1 -- 172.21.15.33:0/2837783121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f27ea0 con 0x560b83fad220 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dff200640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9de8010960 tx=0x7f9de8063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affc1717deb2b6a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8065040 con 0x560b83fad220 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9de8050ba0 con 0x560b83fad220 2024-03-28T01:08:23.897 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9e0324a480 10 client.4698 Subscribing to map 'mdsmap' 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.891+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83eff2f0 con 0x560b83fad220 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9df6000640 20 client.4698 tick 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9df6000640 20 client.4698 collect_and_send_metrics 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9df6000640 20 client.4698 collect_and_send_global_metrics 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9df6000640 5 client.4698 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9df6000640 20 client.4698 trim_cache size 0 max 16384 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9df6000640 20 client.4698 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8085020 con 0x560b83fad220 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8033040 con 0x560b83fad220 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9dfca00640 1 client.4698 handle_mds_map epoch 8 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9e0324a480 20 client.4698 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9e0324a480 10 client.4698 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9e0324a480 20 client.4698 mds is 0 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9e0324a480 10 client.4698 _open_mds_session mds.0 2024-03-28T01:08:23.898 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.899 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.899 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9e0324a480 10 client.4698 waiting for session to mds.0 to open 2024-03-28T01:08:23.899 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.892+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.899 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.893+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e60c1284acff005 server_cookie=5b4a8a4e085348c9 in_seq=0 out_seq=0 2024-03-28T01:08:23.899 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.893+0000 7f9dfca00640 10 client.4698 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9dfca00640 10 client.4698 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9dfca00640 10 client.4698 renew_caps mds.0 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9dfca00640 10 client.4698 connect_mds_targets for mds.0 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 10 client.4698 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 20 client.4698 mds is 0 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 10 client.4698 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 20 client.4698 encode_cap_releases enter (req: 0x560b84053a70, mds: 0) 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 20 client.4698 send_request set sent_stamp to 2024-03-28T01:08:23.897436+0000 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 10 client.4698 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.893024+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.893024+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84053e40 con 0x560b84056300 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9e0324a480 20 client.4698 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:23.901 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.896+0000 7f9dfca00640 10 client.4698 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b84056300 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 insert_trace from 2024-03-28T01:08:23.897436+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 features 0xffffffffffffffff 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 update_snap_trace len 70 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 get_snap_realm 0x1 0x7f9dc40034b0, nref now is 1 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 15 client.4698 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 15 client.4698 snapc 1=[] 2024-03-28T01:08:23.902 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 put_snap_realm 0x1 0x7f9dc40034b0 2 -> 1 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 hrm is_target=1 is_dentry=0 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a7a0) - ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 dir hash is 2 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 12 client.4698 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 15 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 12 client.4698 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) caps pAsLsXsFs 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 get_snap_realm 0x1 0x7f9dc40034b0, nref was 1 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 get_snap_realm 0x1 0x7f9dc40034b0, nref now is 2 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 15 client.4698 add_update_cap first one, opened snaprealm 0x7f9dc40034b0 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 put_snap_realm 0x1 0x7f9dc40034b0 2 -> 1 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:23.903 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 lat 0.000590 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 _ll_get 0x7f9dc400a7a0 0x1 -> 1 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_reclaim(unknownreclaimflag:aef6 flags 0xffffffff) v1 -- 0x560b84053e40 con 0x560b84056300 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reclaim_reply(-22 e 0) v1 ==== 13+0+0 (crc 0 0 0) 0x7f9df00a5080 con 0x560b84056300 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9dfca00640 10 client.4698 handle_client_reclaim_reply client_reclaim_reply(-22 e 0) v1 from mds.0 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 2 client.4698 unmount 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 2 client.4698 unmounting 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 _ll_drop_pins 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 _ll_put 0x7f9dc400a7a0 0x1 1 -> 0 2024-03-28T01:08:23.904 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 _release 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 _flush 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 nothing to flush 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 flush_caps_sync 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 trim_cache size 0 max 16384 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 _put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 8 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.905 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 15 client.4698 remove_cap last one, closing snaprealm 0x7f9dc40034b0 2024-03-28T01:08:23.906 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 20 client.4698 put_snap_realm 0x1 0x7f9dc40034b0 1 -> 0 2024-03-28T01:08:23.906 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 10 client.4698 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.906 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 2 client.4698 _close_mds_session mds.0 seq 0 2024-03-28T01:08:23.906 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.906 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.897+0000 7f9e0324a480 2 client.4698 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:23.907 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.907 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 10 client.4698 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:23.907 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 5 client.4698 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:23.907 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 1 -- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83fa6430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.907 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.907 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 10 client.4698 remove_session_caps mds.0 2024-03-28T01:08:23.907 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 10 client.4698 kick_requests_closed for mds.0 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 10 client.4698 unmounting: trim pass, size was 0+0 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 20 client.4698 trim_cache size 0 max 16384 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9dfca00640 10 client.4698 unmounting: trim pass, size still 0+0 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 2 client.4698 unmounted. 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 client.4698 shutdown 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 msgr2=0x560b83fba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9de8010960 tx=0x7f9de8063000 comp rx=0 tx=0).stop 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 shutdown_connections 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=CLOSED pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83f2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2837783121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 >> 172.21.15.33:0/2837783121 conn(0x560b83f2d970 msgr2=0x560b840566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 shutdown_connections 2024-03-28T01:08:23.908 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 1 -- 172.21.15.33:0/2837783121 wait complete. 2024-03-28T01:08:23.909 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.901+0000 7f9e0324a480 20 client.4698 trim_cache size 0 max 16384 2024-03-28T01:08:23.909 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.CheckNegativeReclaimFlag (13 ms) 2024-03-28T01:08:23.909 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameRemount 2024-03-28T01:08:23.913 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.902+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:23.913 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.902+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:23.913 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.913 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4007f00 con 0x560b83fa9590 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b84053ff0 con 0x560b83fb16a0 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbede0 con 0x560b83fad220 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60918/0 (socket says 172.21.15.33:60918) 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9dffc00640 1 -- 172.21.15.33:0/2520275456 learned_addr learned my addr 172.21.15.33:0/2520275456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9e00600640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.903+0000 7f9dff200640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dffc00640 1 -- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 msgr2=0x560b83fba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dffc00640 1 -- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.914 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dffc00640 1 -- 172.21.15.33:0/2520275456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f2b2d0 con 0x560b83fa9590 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9df000a0b0 tx=0x560b83fb4a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b09cbced42b3b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5040 con 0x560b83fa9590 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df00af7f0 con 0x560b83fa9590 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9e0324a480 10 client.4701 Subscribing to map 'mdsmap' 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83f2b610 con 0x560b83fa9590 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9df5600640 20 client.4701 tick 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9df5600640 20 client.4701 collect_and_send_metrics 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9df5600640 20 client.4701 collect_and_send_global_metrics 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9df5600640 5 client.4701 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9df5600640 20 client.4701 trim_cache size 0 max 16384 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.904+0000 7f9df5600640 20 client.4701 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b84053e40 con 0x560b83fa9590 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0091040 con 0x560b83fa9590 2024-03-28T01:08:23.915 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9dfde00640 1 client.4701 handle_mds_map epoch 8 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9e0324a480 20 client.4701 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9e0324a480 10 client.4701 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9e0324a480 20 client.4701 mds is 0 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9e0324a480 10 client.4701 _open_mds_session mds.0 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9e0324a480 10 client.4701 waiting for session to mds.0 to open 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.905+0000 7f9dff200640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.906+0000 7f9dff200640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f2cb10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d634001987188014 server_cookie=4eac7b828110d285 in_seq=0 out_seq=0 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.906+0000 7f9dfde00640 10 client.4701 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 renew_caps mds.0 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.916 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 connect_mds_targets for mds.0 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 10 client.4701 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 mds is 0 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 10 client.4701 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 encode_cap_releases enter (req: 0x560b83f2e4f0, mds: 0) 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 send_request set sent_stamp to 2024-03-28T01:08:23.909961+0000 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 10 client.4701 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.905873+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.905873+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa66e0 con 0x560b84056300 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a580 con 0x560b84056300 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de802b8b0 con 0x560b84056300 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 insert_trace from 2024-03-28T01:08:23.909961+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:23.917 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 features 0xffffffffffffffff 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 update_snap_trace len 70 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0x7f9dc4004ff0, nref now is 1 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 15 client.4701 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 15 client.4701 snapc 1=[] 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 put_snap_realm 0x1 0x7f9dc4004ff0 2 -> 1 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 hrm is_target=1 is_dentry=0 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a7a0) - ctime 2024-03-28T01:08:18.796876+0000 mtime 2024-03-28T01:08:18.796876+0000 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 dir hash is 2 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 12 client.4701 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 15 2024-03-28T01:08:23.918 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 12 client.4701 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=- has_dir_layout 0x7f9dc400a7a0) caps pAsLsXsFs 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0x7f9dc4004ff0, nref was 1 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0x7f9dc4004ff0, nref now is 2 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 15 client.4701 add_update_cap first one, opened snaprealm 0x7f9dc4004ff0 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 10 client.4701 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 put_snap_realm 0x1 0x7f9dc4004ff0 2 -> 1 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 lat 0.000641 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 _ll_get 0x7f9dc400a7a0 0x1 -> 1 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 10 client.4701 mkdirat: AlternateNameRemount_44790 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9dfde00640 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.919 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 10 client.4701 path_walk 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 20 client.4701 may_create 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 10 client.4701 _getattr mask As issued=1 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 3 client.4701 may_create 0x7f9dc400a7a0 = 0 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.909+0000 7f9e0324a480 8 client.4701 _mkdir(0x1 AlternateNameRemount_44790, 0777, uid 0, gid 0) 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 get_or_create 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) name AlternateNameRemount_44790 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 15 open_dir 0x560b83fa6580 on 0x7f9dc400a7a0 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 15 client.4701 link dir 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 'AlternateNameRemount_44790' dn 0x1.head["AlternateNameRemount_44790"] (new dn) 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 10 client.4701 _mkdir: making request 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 choose_target_mds inode dir hash is 2 on AlternateNameRemount_44790 => 2797286712 2024-03-28T01:08:23.920 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 choose_target_mds 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) is_hash=1 hash=2797286712 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 10 client.4701 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 mds is 0 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 10 client.4701 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 encode_cap_releases enter (req: 0x560b83fa66e0, mds: 0) 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 encode_dentry_release enter(dn:0x7f9dc400a490) 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0), req:0x560b83fa66e0 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 send_request set sent_stamp to 2024-03-28T01:08:23.910721+0000 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 10 client.4701 send_request client_request(unknown.0:2 mkdir owner_uid=0, owner_gid=0 #0x1/AlternateNameRemount_44790 (foo) 2024-03-28T01:08:23.910701+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 mkdir owner_uid=0, owner_gid=0 #0x1/AlternateNameRemount_44790 (foo) 2024-03-28T01:08:23.910701+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f28ca0 con 0x560b84056300 2024-03-28T01:08:23.921 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.910+0000 7f9e0324a480 20 client.4701 awaiting reply|forward|kick on 0x7ffe68269870 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 832+0+0 (crc 0 0 0) 0x7f9de800f4a0 con 0x560b84056300 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 insert_trace from 2024-03-28T01:08:23.910721+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 features 0xffffffffffffffff 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 update_snap_trace len 70 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0x7f9dc4004ff0, nref was 1 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0x7f9dc4004ff0, nref now is 2 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 hrm is_target=1 is_dentry=1 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 update_inode_file_time 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400ad80) - ctime 2024-03-28T01:08:23.910701+0000 mtime 2024-03-28T01:08:23.910701+0000 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 dir hash is 2 2024-03-28T01:08:23.927 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 12 client.4701 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 12 client.4701 add_update_inode adding 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=- 0x7f9dc400ad80) caps pAsxLsXsxFsx 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0x7f9dc4004ff0, nref was 2 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 get_snap_realm 0x1 0x7f9dc4004ff0, nref now is 3 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 15 client.4701 add_update_cap first one, opened snaprealm 0x7f9dc4004ff0 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 add_update_cap issued - -> pAsxLsXsxFsx from mds.0 on 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) 0x7f9dc400ad80) 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) 0x7f9dc400ad80) 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 12 client.4701 add_update_inode had 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7a0) caps pAsLsXsFs 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 update_inode_file_time 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:18.796876+0000 ctime=2024-03-28T01:08:18.796876+0000 change_attr=15 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7a0) pAsLsXs ctime 2024-03-28T01:08:23.910701+0000 mtime 2024-03-28T01:08:23.910701+0000 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 dir hash is 2 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 12 client.4701 add_update_inode client inode change_attr: 15 , mds inodestat change_attr: 16 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.928 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 12 client.4701 insert_dentry_inode 'AlternateNameRemount_44790' vino 0x10000000008.head in dir 0x1.head dn 0x7f9dc400a490 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 15 client.4701 link dir 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 'AlternateNameRemount_44790' to inode 0x7f9dc400ad80 dn 0x1.head["AlternateNameRemount_44790"] (old dn) 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_inode on 0x10000000008.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad80) n = 1 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 link inode 0x7f9dc400ad80 parents now 0x1.head["AlternateNameRemount_44790"] 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_inode on 0x10000000008.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad80) n = 1 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 15 client.4701 update_dentry_lease 0x1.head["AlternateNameRemount_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:23.910721+0000 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_snap_realm 0x1 0x7f9dc4004ff0 3 -> 2 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 handle_client_reply signalling caller 0x7ffe68269870 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 handle_client_reply awaiting kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 sendrecv kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 lat 0.004401 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _mkdir result is 0 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 trim_cache size 1 max 16384 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 8 client.4701 _mkdir(#0x1/AlternateNameRemount_44790, 040777) = 0 2024-03-28T01:08:23.929 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 2 client.4701 unmount 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_inode on 0x10000000008.head(faked_ino=0 nref=5 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad80) n = 1 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 unmounting: trim pass, size was 1+2 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 trim_cache size 1 max 16384 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 15 client.4701 trim_dentry unlinking dn AlternateNameRemount_44790 in dir 0x1 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 15 client.4701 unlink dir 0x7f9dc400a7a0 'AlternateNameRemount_44790' dn 0x7f9dc400a490 inode 0x7f9dc400ad80 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_inode on 0x10000000008.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) n = 1 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 unlink inode 0x7f9dc400ad80 parents now 2024-03-28T01:08:23.930 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 15 client.4701 unlink removing 'AlternateNameRemount_44790' dn 0x7f9dc400a490 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 15 client.4701 close_dir dir 0x560b83fa6580 on 0x7f9dc400a7a0 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 20 client.4701 put_inode on 0x10000000008.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) n = 1 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9dfde00640 10 client.4701 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 2 client.4701 unmounting 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _ll_drop_pins 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 _ll_put 0x7f9dc400a7a0 0x1 1 -> 0 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 _release 0x10000000008.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _invalidate_inode_cache 0x10000000008.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _flush 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) 2024-03-28T01:08:23.931 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 nothing to flush 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) n = 1 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 _release 0x1.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _flush 0x1.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 nothing to flush 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) n = 1 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_inode on 0x1.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 1 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 flush_caps_sync 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 trim_cache size 0 max 16384 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _put_inode on 0x1.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) n = 14 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.932 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 15 client.4701 remove_cap last one, closing snaprealm 0x7f9dc4004ff0 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_snap_realm 0x1 0x7f9dc4004ff0 2 -> 1 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=- has_dir_layout 0x7f9dc400a7a0) 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _put_inode on 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) n = 7 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 remove_cap mds.0 on 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE 0x7f9dc400ad80) 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 15 client.4701 remove_cap last one, closing snaprealm 0x7f9dc4004ff0 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 20 client.4701 put_snap_realm 0x1 0x7f9dc4004ff0 1 -> 0 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 10 client.4701 _put_inode deleting 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=- COMPLETE 0x7f9dc400ad80) 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 2 client.4701 _close_mds_session mds.0 seq 0 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.914+0000 7f9e0324a480 2 client.4701 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 10 client.4701 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 5 client.4701 _closed_mds_session mds.0 seq 0 2024-03-28T01:08:23.933 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 1 -- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83f2cb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f2cb10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 10 client.4701 remove_session_caps mds.0 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 10 client.4701 kick_requests_closed for mds.0 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 10 client.4701 unmounting: trim pass, size was 0+0 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 20 client.4701 trim_cache size 0 max 16384 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9dfde00640 10 client.4701 unmounting: trim pass, size still 0+0 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 2 client.4701 unmounted. 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 client.4701 shutdown 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9df000a0b0 tx=0x560b83fb4a60 comp rx=0 tx=0).stop 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 shutdown_connections 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83f2cb10 unknown :-1 s=CLOSED pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.934 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2520275456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 >> 172.21.15.33:0/2520275456 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 shutdown_connections 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 1 -- 172.21.15.33:0/2520275456 wait complete. 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.918+0000 7f9e0324a480 20 client.4701 trim_cache size 0 max 16384 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.919+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.920+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4003990 con 0x560b83fb16a0 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b84053c00 con 0x560b83fad220 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b840554b0 con 0x560b83fa9590 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.935 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60934/0 (socket says 172.21.15.33:60934) 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e00600640 1 -- 172.21.15.33:0/470957089 learned_addr learned my addr 172.21.15.33:0/470957089 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9dff200640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9dffc00640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.921+0000 7f9e00600640 1 -- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 msgr2=0x560b83fb3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9e00600640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9e00600640 1 -- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 msgr2=0x560b83fba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9e00600640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9e00600640 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83fa9220 con 0x560b83fb16a0 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dff200640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dffc00640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9e00600640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9df800db30 tx=0x7f9df8025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412ca4a6af2a1514 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df804c040 con 0x560b83fb16a0 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df806c080 con 0x560b83fb16a0 2024-03-28T01:08:23.936 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9e0324a480 10 client.4704 Subscribing to map 'mdsmap' 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83eff1f0 con 0x560b83fb16a0 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9df6000640 20 client.4704 tick 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9df6000640 20 client.4704 collect_and_send_metrics 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9df6000640 20 client.4704 collect_and_send_global_metrics 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9df6000640 5 client.4704 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9df6000640 20 client.4704 trim_cache size 0 max 16384 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9df6000640 20 client.4704 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df806b040 con 0x560b83fb16a0 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b83fb16a0 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.922+0000 7f9dfca00640 1 client.4704 handle_mds_map epoch 8 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9e0324a480 20 client.4704 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9e0324a480 10 client.4704 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9e0324a480 20 client.4704 mds is 0 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9e0324a480 10 client.4704 _open_mds_session mds.0 2024-03-28T01:08:23.937 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9e0324a480 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.938 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.938 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9e0324a480 10 client.4704 waiting for session to mds.0 to open 2024-03-28T01:08:23.938 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9dffc00640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.938 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9dffc00640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb4450 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb56cd6172340638 server_cookie=1655b71e517be2e1 in_seq=0 out_seq=0 2024-03-28T01:08:23.938 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.923+0000 7f9dfca00640 10 client.4704 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:23.946 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.946 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:23.946 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 renew_caps mds.0 2024-03-28T01:08:23.946 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a470 con 0x560b84056300 2024-03-28T01:08:23.946 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 connect_mds_targets for mds.0 2024-03-28T01:08:23.946 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 mds is 0 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 encode_cap_releases enter (req: 0x560b83fa6630, mds: 0) 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 send_request set sent_stamp to 2024-03-28T01:08:23.928076+0000 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.923703+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.923703+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa69a0 con 0x560b84056300 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 awaiting reply|forward|kick on 0x7ffe68269a10 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a470 con 0x560b84056300 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df00b5030 con 0x560b84056300 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 insert_trace from 2024-03-28T01:08:23.928076+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:23.947 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 features 0xffffffffffffffff 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 update_snap_trace len 70 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref now is 1 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 15 client.4704 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 15 client.4704 snapc 1=[] 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 put_snap_realm 0x1 0x7f9dc400a690 2 -> 1 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 hrm is_target=1 is_dentry=0 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a780) - ctime 2024-03-28T01:08:23.910701+0000 mtime 2024-03-28T01:08:23.910701+0000 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 dir hash is 2 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 12 client.4704 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 16 2024-03-28T01:08:23.948 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 12 client.4704 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=- has_dir_layout 0x7f9dc400a780) caps pAsLsXsFs 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref was 1 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref now is 2 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 15 client.4704 add_update_cap first one, opened snaprealm 0x7f9dc400a690 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 10 client.4704 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 put_snap_realm 0x1 0x7f9dc400a690 2 -> 1 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 handle_client_reply signalling caller 0x7ffe68269a10 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 lat 0.000493 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 _ll_get 0x7f9dc400a780 0x1 -> 1 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 walk: AlternateNameRemount_44790 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9dfca00640 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.949 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 path_walk cur=0x1.head(faked_ino=0 nref=7 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 path_walk AlternateNameRemount_44790 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 0 0x1.head(faked_ino=0 nref=7 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) AlternateNameRemount_44790 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 (path is AlternateNameRemount_44790) 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 may_lookup 0x1.head(faked_ino=0 nref=7 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 _getattr mask As issued=1 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 3 client.4704 may_lookup 0x7f9dc400a780 = 0 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 _do_lookup on #0x1/AlternateNameRemount_44790 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=8 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 choose_target_mds inode dir hash is 2 on AlternateNameRemount_44790 => 2797286712 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 20 client.4704 choose_target_mds 0x1.head(faked_ino=0 nref=8 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) is_hash=1 hash=2797286712 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.927+0000 7f9e0324a480 10 client.4704 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=8 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 mds is 0 2024-03-28T01:08:23.950 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 encode_cap_releases enter (req: 0x560b83fa69a0, mds: 0) 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 send_request set sent_stamp to 2024-03-28T01:08:23.928680+0000 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 send_request client_request(unknown.0:2 lookup #0x1/AlternateNameRemount_44790 2024-03-28T01:08:23.928638+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/AlternateNameRemount_44790 2024-03-28T01:08:23.928638+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8406a290 con 0x560b84056300 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 awaiting reply|forward|kick on 0x7ffe68269690 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success) v1 ==== 832+0+0 (crc 0 0 0) 0x7f9df0059070 con 0x560b84056300 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 insert_trace from 2024-03-28T01:08:23.928680+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 features 0xffffffffffffffff 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 update_snap_trace len 70 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref was 1 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref now is 2 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 hrm is_target=1 is_dentry=1 2024-03-28T01:08:23.951 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 update_inode_file_time 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400ad60) - ctime 2024-03-28T01:08:23.910701+0000 mtime 2024-03-28T01:08:23.910701+0000 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 dir hash is 2 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 12 client.4704 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 12 client.4704 add_update_inode adding 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=- 0x7f9dc400ad60) caps pAsLsXsFs 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref was 2 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref now is 3 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 15 client.4704 add_update_cap first one, opened snaprealm 0x7f9dc400a690 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) 0x7f9dc400ad60) 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) 0x7f9dc400ad60) 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 12 client.4704 add_update_inode had 0x1.head(faked_ino=0 nref=8 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) caps pAsLsXsFs 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 dir hash is 2 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 12 client.4704 add_update_inode client inode change_attr: 16 , mds inodestat change_attr: 16 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 10 client.4704 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=8 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.952 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 15 open_dir 0x7f9dc4004720 on 0x7f9dc400a780 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 12 client.4704 insert_dentry_inode 'AlternateNameRemount_44790' vino 0x10000000008.head in dir 0x1.head dn 0 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 15 client.4704 link dir 0x1.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 'AlternateNameRemount_44790' to inode 0x10000000008.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE 0x7f9dc400ad60) dn 0x1.head["AlternateNameRemount_44790"] (new dn) 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 link inode 0x7f9dc400ad60 parents now 0x1.head["AlternateNameRemount_44790"] 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 15 client.4704 update_dentry_lease 0x1.head["AlternateNameRemount_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:23.928680+0000 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 put_snap_realm 0x1 0x7f9dc400a690 3 -> 2 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 handle_client_reply signalling caller 0x7ffe68269690 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 make_request target is 0x10000000008.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) 2024-03-28T01:08:23.953 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 lat 0.000595 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 _do_lookup res is 0 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 _lookup have 0x1.head["AlternateNameRemount_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 _lookup 0x1.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) AlternateNameRemount_44790 = 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=9 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9dfca00640 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 path_walk cur=0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 path_walk 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.954 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 may_delete 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780); ; name AlternateNameRemount_44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 _getattr mask As issued=1 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 3 client.4704 may_delete 0x7f9dc400a780 = 0 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 8 client.4704 _rmdir(0x1 AlternateNameRemount_44790 uid 0 gid 0) 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 get_or_create 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) name AlternateNameRemount_44790 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 _lookup have 0x1.head["AlternateNameRemount_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 _lookup 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) AlternateNameRemount_44790 = 0x10000000008.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60) 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 choose_target_mds inode dir hash is 2 on AlternateNameRemount_44790 => 2797286712 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 choose_target_mds 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780) is_hash=1 hash=2797286712 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 choose_target_mds from dirfragtree hash 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 mds is 0 2024-03-28T01:08:23.955 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 encode_cap_releases enter (req: 0x560b8406a290, mds: 0) 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 encode_inode_release enter(in:0x10000000008.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameRemount_44790"] 0x7f9dc400ad60), req:0x560b8406a290 mds:0, drop:Lsx, unless:-, force:0) 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 encode_dentry_release enter(dn:0x7f9dc400b340) 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a780), req:0x560b8406a290 mds:0, drop:XxFs, unless:Fx, force:1) 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 send_request set sent_stamp to 2024-03-28T01:08:23.929448+0000 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 10 client.4704 send_request client_request(unknown.0:3 rmdir #0x1/AlternateNameRemount_44790 2024-03-28T01:08:23.929412+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 rmdir #0x1/AlternateNameRemount_44790 2024-03-28T01:08:23.929412+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8406a680 con 0x560b84056300 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.928+0000 7f9e0324a480 20 client.4704 awaiting reply|forward|kick on 0x7ffe682699b0 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success unsafe) v1 ==== 498+0+0 (crc 0 0 0) 0x7f9df0063700 con 0x560b84056300 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 handle_client_reply got a reply. Safe:0 tid 3 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 10 client.4704 insert_trace from 2024-03-28T01:08:23.929448+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 10 client.4704 features 0xffffffffffffffff 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 10 client.4704 update_snap_trace len 70 2024-03-28T01:08:23.956 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref was 2 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 get_snap_realm 0x1 0x7f9dc400a690, nref now is 3 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 10 client.4704 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 10 client.4704 hrm is_target=0 is_dentry=1 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 12 client.4704 add_update_inode had 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) caps pAsLsXs 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 10 client.4704 update_inode_file_time 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=16 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) pAsLsXs ctime 2024-03-28T01:08:23.929412+0000 mtime 2024-03-28T01:08:23.929412+0000 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 dir hash is 2 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 12 client.4704 add_update_inode client inode change_attr: 16 , mds inodestat change_attr: 17 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 10 client.4704 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 15 client.4704 unlink dir 0x7f9dc400a780 'AlternateNameRemount_44790' dn 0x7f9dc400b340 inode 0x7f9dc400ad60 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 unlink inode 0x7f9dc400ad60 parents now 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.957 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 put_snap_realm 0x1 0x7f9dc400a690 3 -> 2 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.929+0000 7f9dfca00640 20 client.4704 handle_client_reply signalling caller 0x7ffe682699b0 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9dfca00640 20 client.4704 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 20 client.4704 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 20 client.4704 lat 0.001265 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 20 client.4704 trim_cache size 1 max 16384 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 8 client.4704 rmdir(#0x1/AlternateNameRemount_44790) = 0 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 2 client.4704 unmount 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 2 client.4704 unmounting 2024-03-28T01:08:23.958 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.930+0000 7f9e0324a480 10 client.4704 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(grant ino 0x10000000008 27 seq 2 caps=pAsLsXsFsx dirty=- wanted=Ls follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.910701+0000 ctime 2024-03-28T01:08:23.929412+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081040 con 0x560b84056300 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 5 client.4704 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 mds.0 seq now 1 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 5 client.4704 handle_cap_grant on in 0x10000000008 mds.0 seq 2 caps now pAsLsXsFsx was pAsXsFs 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 update_inode_file_time 0x10000000008.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.910701+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400ad60) pAsXsFs ctime 2024-03-28T01:08:23.929412+0000 mtime 2024-03-28T01:08:23.910701+0000 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 grant, new caps are LsFx 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 5 client.4704 _try_to_trim_inode in 0x10000000008.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 unmounting: trim pass, size was 1+2 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 20 client.4704 trim_cache size 1 max 16384 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 unmounting: trim pass, size still 1+2 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df001ebf0 con 0x560b84056300 2024-03-28T01:08:23.959 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 20 client.4704 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 insert_trace from 2024-03-28T01:08:23.929448+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 5 client.4704 _try_to_trim_inode in 0x10000000008.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 unmounting: trim pass, size was 1+2 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 20 client.4704 trim_cache size 1 max 16384 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 15 client.4704 trim_dentry unlinking dn AlternateNameRemount_44790 in dir 0x1 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 15 client.4704 unlink dir 0x7f9dc400a780 'AlternateNameRemount_44790' dn 0x7f9dc400b340 inode 0 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 15 client.4704 unlink removing 'AlternateNameRemount_44790' dn 0x7f9dc400b340 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 15 client.4704 close_dir dir 0x7f9dc4004720 on 0x7f9dc400a780 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9dfca00640 10 client.4704 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:23.960 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _ll_drop_pins 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 _ll_put 0x7f9dc400a780 0x1 1 -> 0 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 _release 0x10000000008.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _invalidate_inode_cache 0x10000000008.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _flush 0x10000000008.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 nothing to flush 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 _release 0x1.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _flush 0x1.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.961 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 nothing to flush 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x10000000008.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) n = 1 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 1 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 flush_caps_sync 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 trim_cache size 0 max 16384 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) n = 16 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 15 client.4704 remove_cap last one, closing snaprealm 0x7f9dc400a690 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_snap_realm 0x1 0x7f9dc400a690 2 -> 1 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=- has_dir_layout 0x7f9dc400a780) 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _put_inode on 0x10000000008.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) n = 12 2024-03-28T01:08:23.962 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 remove_cap mds.0 on 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400ad60) 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 15 client.4704 remove_cap last one, closing snaprealm 0x7f9dc400a690 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 20 client.4704 put_snap_realm 0x1 0x7f9dc400a690 1 -> 0 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 10 client.4704 _put_inode deleting 0x10000000008.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.910701+0000 mtime=2024-03-28T01:08:23.910701+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=1 caps=- COMPLETE 0x7f9dc400ad60) 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 2 client.4704 _close_mds_session mds.0 seq 1 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 1) v7 -- 0x560b83fa9f90 con 0x560b84056300 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.934+0000 7f9e0324a480 2 client.4704 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 10 client.4704 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 5 client.4704 _closed_mds_session mds.0 seq 1 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 1 -- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83fb4450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb4450 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 10 client.4704 remove_session_caps mds.0 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 10 client.4704 kick_requests_closed for mds.0 2024-03-28T01:08:23.963 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 10 client.4704 unmounting: trim pass, size was 0+0 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 20 client.4704 trim_cache size 0 max 16384 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9dfca00640 10 client.4704 unmounting: trim pass, size still 0+0 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 2 client.4704 unmounted. 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 client.4704 shutdown 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 msgr2=0x560b84059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9df800db30 tx=0x7f9df8025000 comp rx=0 tx=0).stop 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 shutdown_connections 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fb4450 unknown :-1 s=CLOSED pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83fba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 --2- 172.21.15.33:0/470957089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fb3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 >> 172.21.15.33:0/470957089 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 shutdown_connections 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 1 -- 172.21.15.33:0/470957089 wait complete. 2024-03-28T01:08:23.964 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.938+0000 7f9e0324a480 20 client.4704 trim_cache size 0 max 16384 2024-03-28T01:08:23.965 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.AlternateNameRemount (37 ms) 2024-03-28T01:08:23.965 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameMkdir 2024-03-28T01:08:23.975 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.939+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b8406a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4002d40 con 0x560b83fb16a0 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f27c70 con 0x560b83fad220 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fa9970 con 0x560b83fa9590 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b8406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b8406a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60948/0 (socket says 172.21.15.33:60948) 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.940+0000 7f9e00600640 1 -- 172.21.15.33:0/3617742621 learned_addr learned my addr 172.21.15.33:0/3617742621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9dff200640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.976 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e00600640 1 -- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 msgr2=0x560b83fba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e00600640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e00600640 1 -- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e00600640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e00600640 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f29dd0 con 0x560b83fb16a0 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e00600640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b8406a190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000fb0 tx=0x7f9df8061000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75062203f761a2f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df804c040 con 0x560b83fb16a0 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df80036b0 con 0x560b83fb16a0 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e0324a480 10 client.4707 Subscribing to map 'mdsmap' 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83ef2b10 con 0x560b83fb16a0 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9df5600640 20 client.4707 tick 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9df5600640 20 client.4707 collect_and_send_metrics 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9df5600640 20 client.4707 collect_and_send_global_metrics 2024-03-28T01:08:23.977 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9df5600640 5 client.4707 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9df5600640 20 client.4707 trim_cache size 0 max 16384 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.941+0000 7f9df5600640 20 client.4707 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8073080 con 0x560b83fb16a0 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009040 con 0x560b83fb16a0 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9dfde00640 1 client.4707 handle_mds_map epoch 8 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9e0324a480 20 client.4707 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9e0324a480 10 client.4707 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9e0324a480 20 client.4707 mds is 0 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9e0324a480 10 client.4707 _open_mds_session mds.0 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9e0324a480 10 client.4707 waiting for session to mds.0 to open 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.942+0000 7f9dfde00640 10 client.4707 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:23.978 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.943+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9017505b951b74a server_cookie=b28db0fe1e89e7c3 in_seq=0 out_seq=0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 renew_caps mds.0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a4c0 con 0x560b84056300 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 connect_mds_targets for mds.0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 mds is 0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 encode_cap_releases enter (req: 0x560b83fa5d10, mds: 0) 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 send_request set sent_stamp to 2024-03-28T01:08:23.947057+0000 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.942885+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.942885+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb3f10 con 0x560b84056300 2024-03-28T01:08:23.979 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a4c0 con 0x560b84056300 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df00b5040 con 0x560b84056300 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 insert_trace from 2024-03-28T01:08:23.947057+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 features 0xffffffffffffffff 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 update_snap_trace len 70 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref now is 1 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 15 client.4707 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 15 client.4707 snapc 1=[] 2024-03-28T01:08:23.980 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 put_snap_realm 0x1 0x7f9dc400a6e0 2 -> 1 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 hrm is_target=1 is_dentry=0 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a7d0) - ctime 2024-03-28T01:08:23.929412+0000 mtime 2024-03-28T01:08:23.929412+0000 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 dir hash is 2 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 12 client.4707 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 17 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 12 client.4707 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=- has_dir_layout 0x7f9dc400a7d0) caps pAsLsXsFs 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref was 1 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref now is 2 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 15 client.4707 add_update_cap first one, opened snaprealm 0x7f9dc400a6e0 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 10 client.4707 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 put_snap_realm 0x1 0x7f9dc400a6e0 2 -> 1 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.981 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 lat 0.000476 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 _ll_get 0x7f9dc400a7d0 0x1 -> 1 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 mkdirat: AlternateNameMkdir_44790 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9dfde00640 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 path_walk 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 may_create 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 _getattr mask As issued=1 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 3 client.4707 may_create 0x7f9dc400a7d0 = 0 2024-03-28T01:08:23.982 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 8 client.4707 _mkdir(0x1 AlternateNameMkdir_44790, 0777, uid 0, gid 0) 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 get_or_create 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) name AlternateNameMkdir_44790 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 15 open_dir 0x560b83fa6b30 on 0x7f9dc400a7d0 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 15 client.4707 link dir 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 'AlternateNameMkdir_44790' dn 0x1.head["AlternateNameMkdir_44790"] (new dn) 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 _mkdir: making request 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 choose_target_mds inode dir hash is 2 on AlternateNameMkdir_44790 => 2481479203 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 choose_target_mds 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) is_hash=1 hash=2481479203 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 mds is 0 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 10 client.4707 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 encode_cap_releases enter (req: 0x560b83fb3f10, mds: 0) 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 encode_dentry_release enter(dn:0x7f9dc4004ff0) 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.946+0000 7f9e0324a480 20 client.4707 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0), req:0x560b83fb3f10 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:23.983 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.947+0000 7f9e0324a480 20 client.4707 send_request set sent_stamp to 2024-03-28T01:08:23.947676+0000 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.947+0000 7f9e0324a480 10 client.4707 send_request client_request(unknown.0:2 mkdir owner_uid=0, owner_gid=0 #0x1/AlternateNameMkdir_44790 (foo) 2024-03-28T01:08:23.947622+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.947+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 mkdir owner_uid=0, owner_gid=0 #0x1/AlternateNameMkdir_44790 (foo) 2024-03-28T01:08:23.947622+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb4400 con 0x560b84056300 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.947+0000 7f9e0324a480 20 client.4707 awaiting reply|forward|kick on 0x7ffe682696c0 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 830+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b84056300 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 insert_trace from 2024-03-28T01:08:23.947676+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 features 0xffffffffffffffff 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 update_snap_trace len 70 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref was 1 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref now is 2 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 hrm is_target=1 is_dentry=1 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x10000000009.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400adb0) - ctime 2024-03-28T01:08:23.947622+0000 mtime 2024-03-28T01:08:23.947622+0000 2024-03-28T01:08:23.984 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 dir hash is 2 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 12 client.4707 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 12 client.4707 add_update_inode adding 0x10000000009.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=- 0x7f9dc400adb0) caps pAsxLsXsxFsx 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref was 2 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref now is 3 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 15 client.4707 add_update_cap first one, opened snaprealm 0x7f9dc400a6e0 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 add_update_cap issued - -> pAsxLsXsxFsx from mds.0 on 0x10000000009.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) 0x7f9dc400adb0) 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x10000000009.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) 0x7f9dc400adb0) 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 12 client.4707 add_update_inode had 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) caps pAsLsXsFs 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.929412+0000 ctime=2024-03-28T01:08:23.929412+0000 change_attr=17 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) pAsLsXs ctime 2024-03-28T01:08:23.947622+0000 mtime 2024-03-28T01:08:23.947622+0000 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 dir hash is 2 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 12 client.4707 add_update_inode client inode change_attr: 17 , mds inodestat change_attr: 18 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 10 client.4707 add_update_cap calling signal_caps_inode 2024-03-28T01:08:23.985 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 12 client.4707 insert_dentry_inode 'AlternateNameMkdir_44790' vino 0x10000000009.head in dir 0x1.head dn 0x7f9dc4004ff0 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 15 client.4707 link dir 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 'AlternateNameMkdir_44790' to inode 0x7f9dc400adb0 dn 0x1.head["AlternateNameMkdir_44790"] (old dn) 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 link inode 0x7f9dc400adb0 parents now 0x1.head["AlternateNameMkdir_44790"] 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 15 client.4707 update_dentry_lease 0x1.head["AlternateNameMkdir_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:23.947676+0000 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 put_snap_realm 0x1 0x7f9dc400a6e0 3 -> 2 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 handle_client_reply signalling caller 0x7ffe682696c0 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 handle_client_reply awaiting kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 sendrecv kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 lat 0.004544 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 _mkdir result is 0 2024-03-28T01:08:23.986 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 trim_cache size 1 max 16384 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 8 client.4707 _mkdir(#0x1/AlternateNameMkdir_44790, 040777) = 0 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 walk: AlternateNameMkdir_44790 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=5 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9dfde00640 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 path_walk cur=0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 path_walk AlternateNameMkdir_44790 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 0 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) AlternateNameMkdir_44790 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 (path is AlternateNameMkdir_44790) 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 may_lookup 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 _getattr mask As issued=1 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 3 client.4707 may_lookup 0x7f9dc400a7d0 = 0 2024-03-28T01:08:23.987 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 _lookup have 0x1.head["AlternateNameMkdir_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 _lookup 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) AlternateNameMkdir_44790 = 0x10000000009.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 path_walk cur=0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 path_walk 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 may_delete 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0); ; name AlternateNameMkdir_44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 _getattr mask As issued=1 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 3 client.4707 may_delete 0x7f9dc400a7d0 = 0 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 8 client.4707 _rmdir(0x1 AlternateNameMkdir_44790 uid 0 gid 0) 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 get_or_create 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) name AlternateNameMkdir_44790 2024-03-28T01:08:23.988 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 _lookup have 0x1.head["AlternateNameMkdir_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 _lookup 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) AlternateNameMkdir_44790 = 0x10000000009.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 choose_target_mds inode dir hash is 2 on AlternateNameMkdir_44790 => 2481479203 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 choose_target_mds 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0) is_hash=1 hash=2481479203 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 choose_target_mds from dirfragtree hash 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 mds is 0 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 encode_cap_releases enter (req: 0x560b83fb4400, mds: 0) 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 encode_inode_release enter(in:0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0), req:0x560b83fb4400 mds:0, drop:Lsx, unless:-, force:0) 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 encode_dentry_release enter(dn:0x7f9dc4004ff0) 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a7d0), req:0x560b83fb4400 mds:0, drop:XxFs, unless:Fx, force:1) 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 send_request set sent_stamp to 2024-03-28T01:08:23.952351+0000 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 10 client.4707 send_request client_request(unknown.0:3 rmdir #0x1/AlternateNameMkdir_44790 2024-03-28T01:08:23.952328+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:23.989 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 rmdir #0x1/AlternateNameMkdir_44790 2024-03-28T01:08:23.952328+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb47c0 con 0x560b84056300 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.951+0000 7f9e0324a480 20 client.4707 awaiting reply|forward|kick on 0x7ffe68269800 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_caps(revoke ino 0x10000000009 1 seq 2 caps=pAsXsFsx dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.947622+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081040 con 0x560b84056300 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 5 client.4707 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 mds.0 seq now 1 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 5 client.4707 handle_cap_grant on in 0x10000000009 mds.0 seq 2 caps now pAsXsFsx was pAsxXsxFsx 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsxXsxFsx(0=pAsxXsxFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) pAsxXsxFsx ctime 2024-03-28T01:08:23.947622+0000 mtime 2024-03-28T01:08:23.947622+0000 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 revocation of AxXx 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 check_caps on 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFsx(0=pAsXsFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) wanted AsLsXsFsx used - issued pAsXsFsx revoking AxXx flags=1 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 cap mds.0 issued pAsXsFsx implemented pAsxXsxFsx revoking AxXx 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 completed revocation of AxXx 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 send_cap 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFsx(0=pAsXsFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) mds.0 seq 2 used - want AsLsXsFsx flush - retain pAsLsXsFsx held pAsxXsxFsx revoking AxXx dropping - 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 15 client.4707 auth cap, requesting max_size 0 2024-03-28T01:08:23.990 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000009 1 seq 2 caps=pAsXsFsx dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.947622+0000 change_attr 0) v12 -- 0x7f9dc400b390 con 0x560b84056300 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(revoke ino 0x10000000009 1 seq 3 caps=pAsXsFs dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.947622+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400b390 con 0x560b84056300 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 mds.0 seq now 2 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 5 client.4707 handle_cap_grant on in 0x10000000009 mds.0 seq 3 caps now pAsXsFs was pAsXsFsx 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFsx(0=pAsXsFsx) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) pAsXsFsx ctime 2024-03-28T01:08:23.947622+0000 mtime 2024-03-28T01:08:23.947622+0000 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 revocation of Fx 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 check_caps on 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) wanted AsLsXsFsx used - issued pAsXsFs revoking Fx flags=1 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 cap mds.0 issued pAsXsFs implemented pAsXsFsx revoking Fx 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 completed revocation of Fx 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 send_cap 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) mds.0 seq 3 used - want AsLsXsFsx flush - retain pAsLsXsFs held pAsXsFsx revoking Fx dropping - 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 15 client.4707 auth cap, requesting max_size 0 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000009 1 seq 3 caps=pAsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.947622+0000 change_attr 0) v12 -- 0x7f9df0081040 con 0x560b84056300 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:23.991 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_caps(grant ino 0x10000000009 1 seq 4 caps=pAsLsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.947622+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0094050 con 0x560b84056300 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 mds.0 seq now 3 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 5 client.4707 handle_cap_grant on in 0x10000000009 mds.0 seq 4 caps now pAsLsXsFs was pAsXsFs 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) pAsXsFs ctime 2024-03-28T01:08:23.947622+0000 mtime 2024-03-28T01:08:23.947622+0000 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 grant, new caps are Ls 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_caps(revoke ino 0x10000000009 1 seq 5 caps=pAsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.947622+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081040 con 0x560b84056300 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 mds.0 seq now 4 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 5 client.4707 handle_cap_grant on in 0x10000000009 mds.0 seq 5 caps now pAsXsFs was pAsLsXsFs 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) pAsLsXsFs ctime 2024-03-28T01:08:23.947622+0000 mtime 2024-03-28T01:08:23.947622+0000 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 revocation of Ls 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 check_caps on 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) wanted AsLsXsFsx used - issued pAsXsFs revoking Ls flags=1 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 cap mds.0 issued pAsXsFs implemented pAsLsXsFs revoking Ls 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 completed revocation of Ls 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 send_cap 0x10000000009.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameMkdir_44790"] 0x7f9dc400adb0) mds.0 seq 5 used - want AsLsXsFsx flush - retain pAsXsFsx held pAsLsXsFs revoking Ls dropping - 2024-03-28T01:08:23.992 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 15 client.4707 auth cap, requesting max_size 0 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000009 1 seq 5 caps=pAsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.947622+0000 change_attr 0) v12 -- 0x7f9df0094050 con 0x560b84056300 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.952+0000 7f9dfde00640 10 client.4707 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:3 = 0 (0) Success unsafe) v1 ==== 496+0+0 (crc 0 0 0) 0x7f9df00634a0 con 0x560b84056300 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 handle_client_reply got a reply. Safe:0 tid 3 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 insert_trace from 2024-03-28T01:08:23.952351+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 features 0xffffffffffffffff 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 update_snap_trace len 70 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref was 2 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 get_snap_realm 0x1 0x7f9dc400a6e0, nref now is 3 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 hrm is_target=0 is_dentry=1 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 12 client.4707 add_update_inode had 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) caps pAsLsXs 2024-03-28T01:08:23.993 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=18 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) pAsLsXs ctime 2024-03-28T01:08:23.952328+0000 mtime 2024-03-28T01:08:23.952328+0000 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 dir hash is 2 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 12 client.4707 add_update_inode client inode change_attr: 18 , mds inodestat change_attr: 19 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 15 client.4707 unlink dir 0x7f9dc400a7d0 'AlternateNameMkdir_44790' dn 0x7f9dc4004ff0 inode 0x7f9dc400adb0 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 unlink inode 0x7f9dc400adb0 parents now 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 put_snap_realm 0x1 0x7f9dc400a6e0 3 -> 2 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 handle_client_reply signalling caller 0x7ffe68269800 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 handle_client_reply awaiting kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 20 client.4707 sendrecv kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 20 client.4707 lat 0.002069 2024-03-28T01:08:23.994 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 20 client.4707 trim_cache size 1 max 16384 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 8 client.4707 rmdir(#0x1/AlternateNameMkdir_44790) = 0 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 2 client.4707 unmount 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 2 client.4707 unmounting 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(grant ino 0x10000000009 1 seq 6 caps=pAsLsXsFsx dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.947622+0000 ctime 2024-03-28T01:08:23.952328+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0094050 con 0x560b84056300 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9e0324a480 10 client.4707 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 mds.0 seq now 5 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 5 client.4707 handle_cap_grant on in 0x10000000009 mds.0 seq 6 caps now pAsLsXsFsx was pAsXsFs 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 update_inode_file_time 0x10000000009.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.947622+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400adb0) pAsXsFs ctime 2024-03-28T01:08:23.952328+0000 mtime 2024-03-28T01:08:23.947622+0000 2024-03-28T01:08:23.995 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 grant, new caps are LsFx 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 5 client.4707 _try_to_trim_inode in 0x10000000009.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 unmounting: trim pass, size was 1+2 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 20 client.4707 trim_cache size 1 max 16384 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.953+0000 7f9dfde00640 10 client.4707 unmounting: trim pass, size still 1+2 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df0063800 con 0x560b84056300 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 20 client.4707 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 10 client.4707 insert_trace from 2024-03-28T01:08:23.952351+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 10 client.4707 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 5 client.4707 _try_to_trim_inode in 0x10000000009.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.996 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 10 client.4707 unmounting: trim pass, size was 1+2 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 20 client.4707 trim_cache size 1 max 16384 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 15 client.4707 trim_dentry unlinking dn AlternateNameMkdir_44790 in dir 0x1 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 15 client.4707 unlink dir 0x7f9dc400a7d0 'AlternateNameMkdir_44790' dn 0x7f9dc4004ff0 inode 0 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 15 client.4707 unlink removing 'AlternateNameMkdir_44790' dn 0x7f9dc4004ff0 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 15 client.4707 close_dir dir 0x560b83fa6b30 on 0x7f9dc400a7d0 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9dfde00640 10 client.4707 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _ll_drop_pins 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 _ll_put 0x7f9dc400a7d0 0x1 1 -> 0 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.997 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 _release 0x10000000009.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _invalidate_inode_cache 0x10000000009.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _flush 0x10000000009.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 nothing to flush 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 _release 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _flush 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 nothing to flush 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x10000000009.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) n = 1 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 1 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 flush_caps_sync 2024-03-28T01:08:23.998 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 trim_cache size 0 max 16384 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) n = 20 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 15 client.4707 remove_cap last one, closing snaprealm 0x7f9dc400a6e0 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_snap_realm 0x1 0x7f9dc400a6e0 2 -> 1 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=- has_dir_layout 0x7f9dc400a7d0) 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _put_inode on 0x10000000009.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) n = 10 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 remove_cap mds.0 on 0x10000000009.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400adb0) 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 15 client.4707 remove_cap last one, closing snaprealm 0x7f9dc400a6e0 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 20 client.4707 put_snap_realm 0x1 0x7f9dc400a6e0 1 -> 0 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 10 client.4707 _put_inode deleting 0x10000000009.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.947622+0000 mtime=2024-03-28T01:08:23.947622+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=1 caps=- COMPLETE 0x7f9dc400adb0) 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 2 client.4707 _close_mds_session mds.0 seq 5 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 5) v7 -- 0x560b8406b0e0 con 0x560b84056300 2024-03-28T01:08:23.999 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.957+0000 7f9e0324a480 2 client.4707 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 <== mds.0 v2:172.21.15.33:6834/2290842450 12 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 10 client.4707 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 5 client.4707 _closed_mds_session mds.0 seq 5 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 1 -- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83fa6430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 10 client.4707 remove_session_caps mds.0 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 10 client.4707 kick_requests_closed for mds.0 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 10 client.4707 unmounting: trim pass, size was 0+0 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 20 client.4707 trim_cache size 0 max 16384 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9dfde00640 10 client.4707 unmounting: trim pass, size still 0+0 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9e0324a480 2 client.4707 unmounted. 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9e0324a480 1 client.4707 shutdown 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.961+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 msgr2=0x560b8406a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.000 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b8406a190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000fb0 tx=0x7f9df8061000 comp rx=0 tx=0).stop 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 shutdown_connections 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=CLOSED pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fb16a0 0x560b8406a190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3617742621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 >> 172.21.15.33:0/3617742621 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 shutdown_connections 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 -- 172.21.15.33:0/3617742621 wait complete. 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 20 client.4707 trim_cache size 0 max 16384 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.AlternateNameMkdir (24 ms) 2024-03-28T01:08:24.001 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameLong 2024-03-28T01:08:24.010 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.962+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:24.010 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.963+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:24.010 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.963+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.010 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.963+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.010 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.963+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fa6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.010 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.963+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4003080 con 0x560b83fa9590 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.963+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f860 con 0x560b83fb16a0 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.963+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f9e0 con 0x560b83fad220 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60960/0 (socket says 172.21.15.33:60960) 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 -- 172.21.15.33:0/277153548 learned_addr learned my addr 172.21.15.33:0/277153548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dff200640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9e00600640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fa6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 -- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 -- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 msgr2=0x560b83fa6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fa6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.964+0000 7f9dffc00640 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f31e30 con 0x560b83fa9590 2024-03-28T01:08:24.011 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dffc00640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9df00a2040 tx=0x560b84054310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e39398c8504d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5040 con 0x560b83fa9590 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df000e000 con 0x560b83fa9590 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 10 client.4710 Subscribing to map 'mdsmap' 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b8406a8a0 con 0x560b83fa9590 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9df6000640 20 client.4710 tick 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9df6000640 20 client.4710 collect_and_send_metrics 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9df6000640 20 client.4710 collect_and_send_global_metrics 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9df6000640 5 client.4710 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9df6000640 20 client.4710 trim_cache size 0 max 16384 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9df6000640 20 client.4710 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b83ef28d0 con 0x560b83fa9590 2024-03-28T01:08:24.012 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0091040 con 0x560b83fa9590 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dfca00640 1 client.4710 handle_mds_map epoch 8 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 20 client.4710 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 10 client.4710 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 20 client.4710 mds is 0 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 10 client.4710 _open_mds_session mds.0 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83ef2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9e0324a480 10 client.4710 waiting for session to mds.0 to open 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.965+0000 7f9dff200640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83ef2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.966+0000 7f9dff200640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83ef2a80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d0778ac36f1bb9 server_cookie=9c60df1fc6304b33 in_seq=0 out_seq=0 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.966+0000 7f9dfca00640 10 client.4710 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9dfca00640 10 client.4710 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:24.013 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9dfca00640 10 client.4710 renew_caps mds.0 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc4007be0 con 0x560b84056300 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9dfca00640 10 client.4710 connect_mds_targets for mds.0 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 10 client.4710 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 20 client.4710 mds is 0 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 10 client.4710 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 20 client.4710 encode_cap_releases enter (req: 0x560b84061990, mds: 0) 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 20 client.4710 send_request set sent_stamp to 2024-03-28T01:08:23.970506+0000 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 10 client.4710 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.966345+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.966345+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb3f10 con 0x560b84056300 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.969+0000 7f9e0324a480 20 client.4710 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc4007be0 con 0x560b84056300 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:24.014 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de800f4a0 con 0x560b84056300 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 insert_trace from 2024-03-28T01:08:23.970506+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 features 0xffffffffffffffff 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 update_snap_trace len 70 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref now is 1 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 15 client.4710 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 15 client.4710 snapc 1=[] 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 put_snap_realm 0x1 0x7f9dc400a7d0 2 -> 1 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 hrm is_target=1 is_dentry=0 2024-03-28T01:08:24.015 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a8c0) - ctime 2024-03-28T01:08:23.952328+0000 mtime 2024-03-28T01:08:23.952328+0000 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 dir hash is 2 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 12 client.4710 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 19 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 12 client.4710 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=- has_dir_layout 0x7f9dc400a8c0) caps pAsLsXsFs 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref was 1 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref now is 2 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 15 client.4710 add_update_cap first one, opened snaprealm 0x7f9dc400a7d0 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 10 client.4710 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 put_snap_realm 0x1 0x7f9dc400a7d0 2 -> 1 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 lat 0.000545 2024-03-28T01:08:24.016 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 _ll_get 0x7f9dc400a8c0 0x1 -> 1 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 10 client.4710 mkdirat: AlternateNameLong_44790 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9dfca00640 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 10 client.4710 path_walk 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 may_create 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 10 client.4710 _getattr mask As issued=1 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 3 client.4710 may_create 0x7f9dc400a8c0 = 0 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 8 client.4710 _mkdir(0x1 AlternateNameLong_44790, 0777, uid 0, gid 0) 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 get_or_create 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) name AlternateNameLong_44790 2024-03-28T01:08:24.017 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 15 open_dir 0x560b8406a470 on 0x7f9dc400a8c0 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 15 client.4710 link dir 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 'AlternateNameLong_44790' dn 0x1.head["AlternateNameLong_44790"] (new dn) 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 10 client.4710 _mkdir: making request 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 choose_target_mds inode dir hash is 2 on AlternateNameLong_44790 => 3539591241 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 choose_target_mds 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) is_hash=1 hash=3539591241 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 10 client.4710 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 mds is 0 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 10 client.4710 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 encode_cap_releases enter (req: 0x560b83fb3f10, mds: 0) 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 encode_dentry_release enter(dn:0x7f9dc400a6e0) 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0), req:0x560b83fb3f10 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 send_request set sent_stamp to 2024-03-28T01:08:23.971171+0000 2024-03-28T01:08:24.018 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 10 client.4710 send_request client_request(unknown.0:2 mkdir owner_uid=0, owner_gid=0 #0x1/AlternateNameLong_44790 (--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------) 2024-03-28T01:08:23.971133+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 mkdir owner_uid=0, owner_gid=0 #0x1/AlternateNameLong_44790 (--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------) 2024-03-28T01:08:23.971133+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb4260 con 0x560b84056300 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.970+0000 7f9e0324a480 20 client.4710 awaiting reply|forward|kick on 0x7ffe68269890 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = 0 (0) Success safe) v1 ==== 5946+0+0 (crc 0 0 0) 0x7f9de8061be0 con 0x560b84056300 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 insert_trace from 2024-03-28T01:08:23.971171+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 features 0xffffffffffffffff 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 update_snap_trace len 70 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref was 1 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref now is 2 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1000000000a.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400c2b0) - ctime 2024-03-28T01:08:23.971133+0000 mtime 2024-03-28T01:08:23.971133+0000 2024-03-28T01:08:24.019 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 dir hash is 2 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 12 client.4710 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 12 client.4710 add_update_inode adding 0x1000000000a.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=- 0x7f9dc400c2b0) caps pAsxLsXsxFsx 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref was 2 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref now is 3 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 15 client.4710 add_update_cap first one, opened snaprealm 0x7f9dc400a7d0 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 add_update_cap issued - -> pAsxLsXsxFsx from mds.0 on 0x1000000000a.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) 0x7f9dc400c2b0) 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 marking (I_COMPLETE|I_DIR_ORDERED) on empty dir 0x1000000000a.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) 0x7f9dc400c2b0) 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 12 client.4710 add_update_inode had 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) caps pAsLsXsFs 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.952328+0000 ctime=2024-03-28T01:08:23.952328+0000 change_attr=19 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) pAsLsXs ctime 2024-03-28T01:08:23.971133+0000 mtime 2024-03-28T01:08:23.971133+0000 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 dir hash is 2 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 12 client.4710 add_update_inode client inode change_attr: 19 , mds inodestat change_attr: 20 2024-03-28T01:08:24.020 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 10 client.4710 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 12 client.4710 insert_dentry_inode 'AlternateNameLong_44790' vino 0x1000000000a.head in dir 0x1.head dn 0x7f9dc400a6e0 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 15 client.4710 link dir 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 'AlternateNameLong_44790' to inode 0x7f9dc400c2b0 dn 0x1.head["AlternateNameLong_44790"] (old dn) 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 link inode 0x7f9dc400c2b0 parents now 0x1.head["AlternateNameLong_44790"] 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 15 client.4710 update_dentry_lease 0x1.head["AlternateNameLong_44790"] lease(mask 0 dur 0 altn -----------------------------------------------------------------------------------------------------------------------------...)) from 2024-03-28T01:08:23.971171+0000 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 put_snap_realm 0x1 0x7f9dc400a7d0 3 -> 2 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 handle_client_reply signalling caller 0x7ffe68269890 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 lat 0.005085 2024-03-28T01:08:24.021 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 _mkdir result is 0 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 trim_cache size 1 max 16384 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 8 client.4710 _mkdir(#0x1/AlternateNameLong_44790, 040777) = 0 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=5 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 walk: AlternateNameLong_44790 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9dfca00640 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=12 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 path_walk cur=0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 path_walk AlternateNameLong_44790 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 0 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) AlternateNameLong_44790 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 (path is AlternateNameLong_44790) 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 may_lookup 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 _getattr mask As issued=1 2024-03-28T01:08:24.022 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 3 client.4710 may_lookup 0x7f9dc400a8c0 = 0 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 _lookup have 0x1.head["AlternateNameLong_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 _lookup 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) AlternateNameLong_44790 = 0x1000000000a.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=13 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 path_walk cur=0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 path_walk 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 may_delete 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0); ; name AlternateNameLong_44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 _getattr mask As issued=1 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 3 client.4710 may_delete 0x7f9dc400a8c0 = 0 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 8 client.4710 _rmdir(0x1 AlternateNameLong_44790 uid 0 gid 0) 2024-03-28T01:08:24.023 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 get_or_create 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) name AlternateNameLong_44790 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 _lookup have 0x1.head["AlternateNameLong_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 _lookup 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) AlternateNameLong_44790 = 0x1000000000a.head(faked_ino=0 nref=7 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 choose_target_mds inode dir hash is 2 on AlternateNameLong_44790 => 3539591241 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 choose_target_mds 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0) is_hash=1 hash=3539591241 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 choose_target_mds from dirfragtree hash 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 mds is 0 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 encode_cap_releases enter (req: 0x560b83fb4240, mds: 0) 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 encode_inode_release enter(in:0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxLsXsxFsx(0=pAsxLsXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0), req:0x560b83fb4240 mds:0, drop:Lsx, unless:-, force:0) 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 encode_dentry_release enter(dn:0x7f9dc400a6e0) 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a8c0), req:0x560b83fb4240 mds:0, drop:XxFs, unless:Fx, force:1) 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 send_request set sent_stamp to 2024-03-28T01:08:23.976553+0000 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 10 client.4710 send_request client_request(unknown.0:3 rmdir #0x1/AlternateNameLong_44790 2024-03-28T01:08:23.976492+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.024 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 rmdir #0x1/AlternateNameLong_44790 2024-03-28T01:08:23.976492+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb45b0 con 0x560b84056300 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.975+0000 7f9e0324a480 20 client.4710 awaiting reply|forward|kick on 0x7ffe682699d0 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_caps(revoke ino 0x1000000000a 1 seq 2 caps=pAsXsFsx dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.971133+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de804e020 con 0x560b84056300 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 5 client.4710 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 mds.0 seq now 1 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 5 client.4710 handle_cap_grant on in 0x1000000000a mds.0 seq 2 caps now pAsXsFsx was pAsxXsxFsx 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsxXsxFsx(0=pAsxXsxFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) pAsxXsxFsx ctime 2024-03-28T01:08:23.971133+0000 mtime 2024-03-28T01:08:23.971133+0000 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 revocation of AxXx 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 check_caps on 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFsx(0=pAsXsFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) wanted AsLsXsFsx used - issued pAsXsFsx revoking AxXx flags=1 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 cap mds.0 issued pAsXsFsx implemented pAsxXsxFsx revoking AxXx 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 completed revocation of AxXx 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 send_cap 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFsx(0=pAsXsFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) mds.0 seq 2 used - want AsLsXsFsx flush - retain pAsLsXsFsx held pAsxXsxFsx revoking AxXx dropping - 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 15 client.4710 auth cap, requesting max_size 0 2024-03-28T01:08:24.025 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x1000000000a 1 seq 2 caps=pAsXsFsx dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.971133+0000 change_attr 0) v12 -- 0x7f9dc400c9a0 con 0x560b84056300 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(revoke ino 0x1000000000a 1 seq 3 caps=pAsXsFs dirty=- wanted=- follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.971133+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400c9a0 con 0x560b84056300 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 mds.0 seq now 2 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 5 client.4710 handle_cap_grant on in 0x1000000000a mds.0 seq 3 caps now pAsXsFs was pAsXsFsx 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFsx(0=pAsXsFsx) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) pAsXsFsx ctime 2024-03-28T01:08:23.971133+0000 mtime 2024-03-28T01:08:23.971133+0000 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 revocation of Fx 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 check_caps on 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) wanted AsLsXsFsx used - issued pAsXsFs revoking Fx flags=1 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 cap mds.0 issued pAsXsFs implemented pAsXsFsx revoking Fx 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 completed revocation of Fx 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 send_cap 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) mds.0 seq 3 used - want AsLsXsFsx flush - retain pAsLsXsFs held pAsXsFsx revoking Fx dropping - 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 15 client.4710 auth cap, requesting max_size 0 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x1000000000a 1 seq 3 caps=pAsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.971133+0000 change_attr 0) v12 -- 0x7f9de804e020 con 0x560b84056300 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.026 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_caps(grant ino 0x1000000000a 1 seq 4 caps=pAsLsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.971133+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de800a070 con 0x560b84056300 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 mds.0 seq now 3 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 5 client.4710 handle_cap_grant on in 0x1000000000a mds.0 seq 4 caps now pAsLsXsFs was pAsXsFs 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) pAsXsFs ctime 2024-03-28T01:08:23.971133+0000 mtime 2024-03-28T01:08:23.971133+0000 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 grant, new caps are Ls 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.976+0000 7f9dfca00640 10 client.4710 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_caps(revoke ino 0x1000000000a 1 seq 5 caps=pAsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.971133+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de804e020 con 0x560b84056300 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 mds.0 seq now 4 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 5 client.4710 handle_cap_grant on in 0x1000000000a mds.0 seq 5 caps now pAsXsFs was pAsLsXsFs 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) pAsLsXsFs ctime 2024-03-28T01:08:23.971133+0000 mtime 2024-03-28T01:08:23.971133+0000 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 revocation of Ls 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 check_caps on 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) wanted AsLsXsFsx used - issued pAsXsFs revoking Ls flags=1 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 cap mds.0 issued pAsXsFs implemented pAsLsXsFs revoking Ls 2024-03-28T01:08:24.027 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 completed revocation of Ls 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 send_cap 0x1000000000a.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE parents=0x1.head["AlternateNameLong_44790"] 0x7f9dc400c2b0) mds.0 seq 5 used - want AsLsXsFsx flush - retain pAsXsFsx held pAsLsXsFs revoking Ls dropping - 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 15 client.4710 auth cap, requesting max_size 0 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x1000000000a 1 seq 5 caps=pAsXsFs dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.971133+0000 change_attr 0) v12 -- 0x7f9de800a070 con 0x560b84056300 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 10 client.4710 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.977+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:3 = 0 (0) Success unsafe) v1 ==== 5612+0+0 (crc 0 0 0) 0x7f9de8061dd0 con 0x560b84056300 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 handle_client_reply got a reply. Safe:0 tid 3 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 insert_trace from 2024-03-28T01:08:23.976553+0000 mds.0 is_target=0 is_dentry=1 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 features 0xffffffffffffffff 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 update_snap_trace len 70 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref was 2 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 get_snap_realm 0x1 0x7f9dc400a7d0, nref now is 3 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 hrm is_target=0 is_dentry=1 2024-03-28T01:08:24.028 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 12 client.4710 add_update_inode had 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) caps pAsLsXs 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=20 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) pAsLsXs ctime 2024-03-28T01:08:23.976492+0000 mtime 2024-03-28T01:08:23.976492+0000 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 dir hash is 2 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 12 client.4710 add_update_inode client inode change_attr: 20 , mds inodestat change_attr: 21 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 15 client.4710 unlink dir 0x7f9dc400a8c0 'AlternateNameLong_44790' dn 0x7f9dc400a6e0 inode 0x7f9dc400c2b0 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 unlink inode 0x7f9dc400c2b0 parents now 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 put_snap_realm 0x1 0x7f9dc400a7d0 3 -> 2 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 handle_client_reply signalling caller 0x7ffe682699d0 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 20 client.4710 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:24.029 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 20 client.4710 lat 0.002269 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 20 client.4710 trim_cache size 1 max 16384 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 8 client.4710 rmdir(#0x1/AlternateNameLong_44790) = 0 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 2 client.4710 unmount 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 2 client.4710 unmounting 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9e0324a480 10 client.4710 waiting for write request '3' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, size was 1+2 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 trim_cache size 1 max 16384 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, size still 1+2 2024-03-28T01:08:24.030 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(grant ino 0x1000000000a 1 seq 6 caps=pAsLsXsFsx dirty=- wanted=AsLsXsFsx follows 0 size 0/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:23.971133+0000 ctime 2024-03-28T01:08:23.976492+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de800a070 con 0x560b84056300 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 mds.0 seq now 5 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 5 client.4710 handle_cap_grant on in 0x1000000000a mds.0 seq 6 caps now pAsLsXsFsx was pAsXsFs 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 update_inode_file_time 0x1000000000a.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.971133+0000 change_attr=0 caps=pAsXsFs(0=pAsXsFs) COMPLETE 0x7f9dc400c2b0) pAsXsFs ctime 2024-03-28T01:08:23.976492+0000 mtime 2024-03-28T01:08:23.971133+0000 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 grant, new caps are LsFx 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 5 client.4710 _try_to_trim_inode in 0x1000000000a.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, size was 1+2 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 20 client.4710 trim_cache size 1 max 16384 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.978+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, size still 1+2 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de802b8b0 con 0x560b84056300 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 20 client.4710 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 10 client.4710 insert_trace from 2024-03-28T01:08:23.976553+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.031 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 10 client.4710 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 5 client.4710 _try_to_trim_inode in 0x1000000000a.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, size was 1+2 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 20 client.4710 trim_cache size 1 max 16384 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 15 client.4710 trim_dentry unlinking dn AlternateNameLong_44790 in dir 0x1 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 15 client.4710 unlink dir 0x7f9dc400a8c0 'AlternateNameLong_44790' dn 0x7f9dc400a6e0 inode 0 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 15 client.4710 unlink removing 'AlternateNameLong_44790' dn 0x7f9dc400a6e0 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 15 client.4710 close_dir dir 0x560b8406a470 on 0x7f9dc400a8c0 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _ll_drop_pins 2024-03-28T01:08:24.032 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 _ll_put 0x7f9dc400a8c0 0x1 1 -> 0 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 _release 0x1000000000a.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _invalidate_inode_cache 0x1000000000a.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _flush 0x1000000000a.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 nothing to flush 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 _release 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _flush 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.033 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 nothing to flush 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1000000000a.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) n = 1 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 1 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 flush_caps_sync 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 wait_sync_caps want 1 (last is 1, 0 total flushing) 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 trim_cache size 0 max 16384 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) n = 20 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 15 client.4710 remove_cap last one, closing snaprealm 0x7f9dc400a7d0 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_snap_realm 0x1 0x7f9dc400a7d0 2 -> 1 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=- has_dir_layout 0x7f9dc400a8c0) 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _put_inode on 0x1000000000a.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) n = 10 2024-03-28T01:08:24.034 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 remove_cap mds.0 on 0x1000000000a.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE 0x7f9dc400c2b0) 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 15 client.4710 remove_cap last one, closing snaprealm 0x7f9dc400a7d0 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 20 client.4710 put_snap_realm 0x1 0x7f9dc400a7d0 1 -> 0 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 10 client.4710 _put_inode deleting 0x1000000000a.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40777 size=0/0 nlink=0 btime=2024-03-28T01:08:23.971133+0000 mtime=2024-03-28T01:08:23.971133+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=1 caps=- COMPLETE 0x7f9dc400c2b0) 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 2 client.4710 _close_mds_session mds.0 seq 5 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 5) v7 -- 0x560b8406a810 con 0x560b84056300 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.982+0000 7f9e0324a480 2 client.4710 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 <== mds.0 v2:172.21.15.33:6834/2290842450 12 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 10 client.4710 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 5 client.4710 _closed_mds_session mds.0 seq 5 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 1 -- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83ef2a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83ef2a80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 10 client.4710 remove_session_caps mds.0 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 10 client.4710 kick_requests_closed for mds.0 2024-03-28T01:08:24.035 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, size was 0+0 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 20 client.4710 trim_cache size 0 max 16384 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9dfca00640 10 client.4710 unmounting: trim pass, size still 0+0 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9e0324a480 2 client.4710 unmounted. 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9e0324a480 1 client.4710 shutdown 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 msgr2=0x560b83fba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.986+0000 7f9e0324a480 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9df00a2040 tx=0x560b84054310 comp rx=0 tx=0).stop 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 shutdown_connections 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83ef2a80 unknown :-1 s=CLOSED pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fb16a0 0x560b83fa6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 --2- 172.21.15.33:0/277153548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 >> 172.21.15.33:0/277153548 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 shutdown_connections 2024-03-28T01:08:24.036 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 1 -- 172.21.15.33:0/277153548 wait complete. 2024-03-28T01:08:24.037 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.987+0000 7f9e0324a480 20 client.4710 trim_cache size 0 max 16384 2024-03-28T01:08:24.037 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.AlternateNameLong (25 ms) 2024-03-28T01:08:24.037 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameCreat 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.988+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.989+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83ee4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4008450 con 0x560b83fa9590 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f30a50 con 0x560b83fad220 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fa9b40 con 0x560b83fb16a0 2024-03-28T01:08:24.045 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60968/0 (socket says 172.21.15.33:60968) 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 -- 172.21.15.33:0/763563792 learned_addr learned my addr 172.21.15.33:0/763563792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dff200640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83ee4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e00600640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 -- 172.21.15.33:0/763563792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fb16a0 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 -- 172.21.15.33:0/763563792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 msgr2=0x560b83ee4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fad220 0x560b83ee4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9dffc00640 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f2b610 con 0x560b83fa9590 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.990+0000 7f9e0324a480 10 client.4713 Subscribing to map 'mdsmap' 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mdsmap=0+}) v3 -- 0x560b8406ccb0 con 0x560b83fa9590 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9dffc00640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9df00033e0 tx=0x560b83fa6970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dffb453855ac62 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9dfde00640 10 client.4713 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:24.046 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9df5600640 20 client.4713 tick 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9df5600640 20 client.4713 collect_and_send_metrics 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9df5600640 20 client.4713 collect_and_send_global_metrics 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9df5600640 5 client.4713 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9df5600640 20 client.4713 trim_cache size 0 max 16384 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9df5600640 20 client.4713 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5070 con 0x560b83fa9590 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df00aabf0 con 0x560b83fa9590 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df0091070 con 0x560b83fa9590 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9dfde00640 1 client.4713 handle_mds_map epoch 8 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 20 client.4713 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 10 client.4713 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 20 client.4713 mds is 0 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 10 client.4713 _open_mds_session mds.0 2024-03-28T01:08:24.047 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9e0324a480 10 client.4713 waiting for session to mds.0 to open 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.991+0000 7f9dff200640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.992+0000 7f9dff200640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2657e96911daddfb server_cookie=1fcf6458524cad5d in_seq=0 out_seq=0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.992+0000 7f9dfde00640 10 client.4713 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056300 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 10 client.4713 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 10 client.4713 renew_caps mds.0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc40039a0 con 0x560b84056300 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 10 client.4713 connect_mds_targets for mds.0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 10 client.4713 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 20 client.4713 mds is 0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 10 client.4713 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:24.048 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 20 client.4713 encode_cap_releases enter (req: 0x560b8406a5c0, mds: 0) 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 20 client.4713 send_request set sent_stamp to 2024-03-28T01:08:23.996252+0000 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 10 client.4713 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.992185+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:23.992185+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83ef2a40 con 0x560b84056300 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9e0324a480 20 client.4713 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc40039a0 con 0x560b84056300 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 10 client.4713 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.995+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de800f4a0 con 0x560b84056300 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 insert_trace from 2024-03-28T01:08:23.996252+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 features 0xffffffffffffffff 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 update_snap_trace len 70 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:24.049 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0x7f9dc400a8c0, nref now is 1 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 15 client.4713 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 15 client.4713 snapc 1=[] 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 put_snap_realm 0x1 0x7f9dc400a8c0 2 -> 1 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 hrm is_target=1 is_dentry=0 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400a9b0) - ctime 2024-03-28T01:08:23.976492+0000 mtime 2024-03-28T01:08:23.976492+0000 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 dir hash is 2 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 12 client.4713 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 21 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 12 client.4713 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=- has_dir_layout 0x7f9dc400a9b0) caps pAsLsXsFs 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0x7f9dc400a8c0, nref was 1 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0x7f9dc400a8c0, nref now is 2 2024-03-28T01:08:24.050 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 15 client.4713 add_update_cap first one, opened snaprealm 0x7f9dc400a8c0 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 put_snap_realm 0x1 0x7f9dc400a8c0 2 -> 1 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 lat 0.000501 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 _ll_get 0x7f9dc400a9b0 0x1 -> 1 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 3 client.4713 openat enter(AlternateNameCreat_44790) 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 path_walk AlternateNameCreat_44790 2024-03-28T01:08:24.051 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 0 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) AlternateNameCreat_44790 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 (path is AlternateNameCreat_44790) 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 may_lookup 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 _getattr mask As issued=1 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 3 client.4713 may_lookup 0x7f9dc400a9b0 = 0 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 _do_lookup on #0x1/AlternateNameCreat_44790 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 choose_target_mds inode dir hash is 2 on AlternateNameCreat_44790 => 2106556820 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 choose_target_mds 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) is_hash=1 hash=2106556820 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 mds is 0 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 encode_cap_releases enter (req: 0x560b83ef2a40, mds: 0) 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 send_request set sent_stamp to 2024-03-28T01:08:23.996815+0000 2024-03-28T01:08:24.052 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 send_request client_request(unknown.0:2 lookup #0x1/AlternateNameCreat_44790 2024-03-28T01:08:23.996798+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/AlternateNameCreat_44790 2024-03-28T01:08:23.996798+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2ca90 con 0x560b84056300 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 awaiting reply|forward|kick on 0x7ffe68269390 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de802bac0 con 0x560b84056300 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 insert_trace from 2024-03-28T01:08:23.996815+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 10 client.4713 insert_trace -- no trace 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 handle_client_reply signalling caller 0x7ffe68269390 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 handle_client_reply awaiting kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 sendrecv kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 lat 0.000346 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 _do_lookup res is -2 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 _lookup 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) AlternateNameCreat_44790 = -2 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.053 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 path_walk cur=0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 path_walk 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 may_create 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 _getattr mask As issued=1 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 3 client.4713 may_create 0x7f9dc400a9b0 = 0 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 8 client.4713 _create(0x1 AlternateNameCreat_44790, 0777) 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 get_or_create 0x1.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) name AlternateNameCreat_44790 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 15 open_dir 0x560b8406a550 on 0x7f9dc400a9b0 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 15 client.4713 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 'AlternateNameCreat_44790' dn 0x1.head["AlternateNameCreat_44790"] (new dn) 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.054 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 choose_target_mds inode dir hash is 2 on AlternateNameCreat_44790 => 2106556820 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 choose_target_mds 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) is_hash=1 hash=2106556820 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 mds is 0 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 encode_cap_releases enter (req: 0x560b83f2ca90, mds: 0) 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 encode_dentry_release enter(dn:0x7f9dc400a7d0) 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0), req:0x560b83f2ca90 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 send_request set sent_stamp to 2024-03-28T01:08:23.997233+0000 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 10 client.4713 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameCreat_44790 (foo) 2024-03-28T01:08:23.997214+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameCreat_44790 (foo) 2024-03-28T01:08:23.997214+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb3f10 con 0x560b84056300 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9e0324a480 20 client.4713 awaiting reply|forward|kick on 0x7ffe682696f0 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:23.996+0000 7f9dfde00640 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 848+0+0 (crc 0 0 0) 0x7f9de800f830 con 0x560b84056300 2024-03-28T01:08:24.055 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 insert_trace from 2024-03-28T01:08:23.997233+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 features 0xffffffffffffffff 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 update_snap_trace len 70 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0x7f9dc400a8c0, nref was 1 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0x7f9dc400a8c0, nref now is 2 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 update_inode_file_time 0x1000000000b.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400af90) - ctime 2024-03-28T01:08:23.997214+0000 mtime 2024-03-28T01:08:23.997214+0000 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 12 client.4713 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 12 client.4713 add_update_inode adding 0x1000000000b.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=- objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400af90) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0x7f9dc400a8c0, nref was 2 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 get_snap_realm 0x1 0x7f9dc400a8c0, nref now is 3 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 15 client.4713 add_update_cap first one, opened snaprealm 0x7f9dc400a8c0 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x1000000000b.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400af90) 2024-03-28T01:08:24.056 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 12 client.4713 add_update_inode had 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a9b0) caps pAsLsXsFs 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 update_inode_file_time 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.976492+0000 ctime=2024-03-28T01:08:23.976492+0000 change_attr=21 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400a9b0) pAsLsXs ctime 2024-03-28T01:08:23.997214+0000 mtime 2024-03-28T01:08:23.997214+0000 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 dir hash is 2 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 12 client.4713 add_update_inode client inode change_attr: 21 , mds inodestat change_attr: 22 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 10 client.4713 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 12 client.4713 insert_dentry_inode 'AlternateNameCreat_44790' vino 0x1000000000b.head in dir 0x1.head dn 0x7f9dc400a7d0 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 15 client.4713 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 'AlternateNameCreat_44790' to inode 0x7f9dc400af90 dn 0x1.head["AlternateNameCreat_44790"] (old dn) 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 link inode 0x7f9dc400af90 parents now 0x1.head["AlternateNameCreat_44790"] 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 15 client.4713 update_dentry_lease 0x1.head["AlternateNameCreat_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:23.997233+0000 2024-03-28T01:08:24.057 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 put_snap_realm 0x1 0x7f9dc400a8c0 3 -> 2 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 handle_client_reply signalling caller 0x7ffe682696f0 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 handle_client_reply awaiting kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 sendrecv kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 client.4713 delegated_inos: [] 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 client.4713 make_request created ino 0x1000000000b 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 make_request target is 0x1000000000b.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 lat 0.004667 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x1000000000b.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x1000000000b.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 client.4713 _create_fh 0x1000000000b mode 2 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 trim_cache size 1 max 16384 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 8 client.4713 create(#0x1/AlternateNameCreat_44790, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:24.058 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 3 client.4713 openat exit(AlternateNameCreat_44790) 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 client.4713 get_quota_root realm 0x1 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 client.4713 get_quota_root 0x1000000000b.head -> 0x1.head 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 10 client.4713 cur file size is 0 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fb47a0 0x560b83f291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- osd_op(unknown.0.0:1 3.29 3:9752f43f:::1000000000b.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f296f0 con 0x560b83fb47a0 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- osd_op(unknown.0.0:2 3.29 3:9752f43f:::1000000000b.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b8404e980 con 0x560b83fb47a0 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9dfde00640 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.001+0000 7f9e00600640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fb47a0 0x560b83f291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.002+0000 7f9dfde00640 10 client.4713 ms_handle_connect on v2:172.21.15.33:6800/2717396394 2024-03-28T01:08:24.059 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.002+0000 7f9e00600640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fb47a0 0x560b83f291b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=542159096234f624 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.002+0000 7f9e00600640 1 -- 172.21.15.33:0/763563792 <== osd.3 v2:172.21.15.33:6800/2717396394 1 ==== osd_op_reply(1 1000000000b.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8065030 con 0x560b83fb47a0 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e00600640 1 -- 172.21.15.33:0/763563792 <== osd.3 v2:172.21.15.33:6800/2717396394 2 ==== osd_op_reply(2 1000000000b.00000000 [create] v23'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8065030 con 0x560b83fb47a0 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 wanted_max_size 0 -> 3 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 get_caps 0x1000000000b.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 clear_suid_sgid 0x1000000000b.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 5 client.4713 get_cap_ref got first FILE_BUFFER ref on 0x1000000000b.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 5 client.4713 get_cap_ref got first FILE_CACHE ref on 0x1000000000b.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000b.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) - -> Fw 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 cap_delay_requeue on 0x1000000000b.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 get_quota_root realm 0x1 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 get_quota_root 0x1000000000b.head -> 0x1.head 2024-03-28T01:08:24.060 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 7 client.4713 wrote to 3, extending file size 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000b.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) Fw -> Fw 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 cap_delay_requeue on 0x1000000000b.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 3 client.4713 write(10, "...", 3, 0) = 3 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 3 client.4713 close enter(10) 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 8 client.4713 _release_fh 0x560b83f2cea0 mode 2 on 0x1000000000b.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _flush 0x1000000000b.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- osd_op(unknown.0.0:3 3.29 3:9752f43f:::1000000000b.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f296f0 con 0x560b83fb47a0 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 check_caps on 0x1000000000b.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 delaying cap release 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 cap_delay_requeue on 0x1000000000b.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _release_fh 0x560b83f2cea0 on inode 0x1000000000b.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) no async_err state 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 3 client.4713 close exit(10) 2024-03-28T01:08:24.061 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 walk: AlternateNameCreat_44790 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 path_walk cur=0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 path_walk AlternateNameCreat_44790 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 0 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) AlternateNameCreat_44790 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 (path is AlternateNameCreat_44790) 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 may_lookup 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _getattr mask As issued=1 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 3 client.4713 may_lookup 0x7f9dc400a9b0 = 0 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 _lookup have 0x1.head["AlternateNameCreat_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _lookup 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) AlternateNameCreat_44790 = 0x1000000000b.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 2 client.4713 unmount 2024-03-28T01:08:24.062 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 2 client.4713 unmounting 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _ll_drop_pins 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 _ll_put 0x7f9dc400a9b0 0x1 1 -> 0 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 _release 0x1000000000b.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _flush 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 _release 0x1.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 _flush 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 nothing to flush 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.063 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 flush_caps_sync 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 check_caps on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=3 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 mark_caps_flushing Fw 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 mark_caps_clean 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 send_cap 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) mds.0 seq 1 used Fcb want Fxcb flush Fw retain pFxcb held pAsxLsXsxFsxcrwb revoking - dropping AsxLsXsxFsrw 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 15 client.4713 auth cap, requesting max_size 3 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x1000000000b 1 seq 1 tid 2 caps=pFxcb dirty=Fw wanted=Fxcb follows 1 size 3/3 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.006090+0000 ctime 2024-03-28T01:08:24.006090+0000 change_attr 1) v12 -- 0x560b8404ee60 con 0x560b84056300 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 wait_sync_caps want 2 (last is 2, 1 total flushing) 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.005+0000 7f9e0324a480 10 client.4713 waiting on mds.0 tid 2 (want 2) 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.008+0000 7f9e00600640 1 -- 172.21.15.33:0/763563792 <== osd.3 v2:172.21.15.33:6800/2717396394 3 ==== osd_op_reply(3 1000000000b.00000000 [write 0~3] v23'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8065030 con 0x560b83fb47a0 2024-03-28T01:08:24.064 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.008+0000 7f9df7400640 10 client.4713 _flushed 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.008+0000 7f9df7400640 5 client.4713 put_cap_ref dropped last FILE_BUFFER ref on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.008+0000 7f9df7400640 5 client.4713 put_cap_ref dropped last FILE_CACHE ref on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.008+0000 7f9df7400640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 2 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.008+0000 7f9df7400640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.008+0000 7f9df7400640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(flush_ack ino 0x1000000000b 1 seq 1 tid 2 caps=pFxcb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x560b8404ee60 con 0x560b84056300 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 5 client.4713 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 10 client.4713 mds.0 seq now 1 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 5 client.4713 handle_cap_flush_ack mds.0 cleaned Fw on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) flushing_caps=Fw objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) with Fw 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 5 client.4713 flushing_caps Fw -> - 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 10 client.4713 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) !flushing 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 10 client.4713 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameCreat_44790"] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.065 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 10 client.4713 unmounting: trim pass, size was 1+2 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 20 client.4713 trim_cache size 1 max 16384 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 15 client.4713 trim_dentry unlinking dn AlternateNameCreat_44790 in dir 0x1 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 15 client.4713 unlink dir 0x7f9dc400a9b0 'AlternateNameCreat_44790' dn 0x7f9dc400a7d0 inode 0x7f9dc400af90 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 20 client.4713 unlink inode 0x7f9dc400af90 parents now 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 15 client.4713 unlink removing 'AlternateNameCreat_44790' dn 0x7f9dc400a7d0 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 15 client.4713 close_dir dir 0x560b8406a550 on 0x7f9dc400a9b0 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 20 client.4713 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 1 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 20 client.4713 put_inode on 0x1000000000b.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400af90) n = 1 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9dfde00640 10 client.4713 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 10 client.4713 wait_sync_caps want 2 (last is 2, 0 total flushing) 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 20 client.4713 trim_cache size 0 max 16384 2024-03-28T01:08:24.066 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 10 client.4713 _put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) n = 19 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 10 client.4713 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 15 client.4713 remove_cap last one, closing snaprealm 0x7f9dc400a8c0 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 20 client.4713 put_snap_realm 0x1 0x7f9dc400a8c0 2 -> 1 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 10 client.4713 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=- has_dir_layout 0x7f9dc400a9b0) 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 10 client.4713 _put_inode on 0x1000000000b.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400af90) n = 15 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 10 client.4713 remove_cap mds.0 on 0x1000000000b.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=pFxcb(0=pFxcb) objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400af90) 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 15 client.4713 remove_cap last one, closing snaprealm 0x7f9dc400a8c0 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 20 client.4713 put_snap_realm 0x1 0x7f9dc400a8c0 1 -> 0 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 10 client.4713 _put_inode deleting 0x1000000000b.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:23.997214+0000 mtime=2024-03-28T01:08:24.006090+0000 ctime=2024-03-28T01:08:24.006090+0000 change_attr=1 caps=- objectset[0x1000000000b ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400af90) 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 2 client.4713 _close_mds_session mds.0 seq 1 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 1) v7 -- 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.009+0000 7f9e0324a480 2 client.4713 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056300 2024-03-28T01:08:24.067 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 10 client.4713 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 5 client.4713 _closed_mds_session mds.0 seq 1 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 1 -- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 msgr2=0x560b83fa6430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056300 0x560b83fa6430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 10 client.4713 remove_session_caps mds.0 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 10 client.4713 kick_requests_closed for mds.0 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 10 client.4713 unmounting: trim pass, size was 0+0 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 20 client.4713 trim_cache size 0 max 16384 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9dfde00640 10 client.4713 unmounting: trim pass, size still 0+0 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9e0324a480 2 client.4713 unmounted. 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.012+0000 7f9e0324a480 1 client.4713 shutdown 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fb47a0 msgr2=0x560b83f291b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x560b83fb47a0 0x560b83f291b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 msgr2=0x560b83fba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.068 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 --2- 172.21.15.33:0/763563792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9590 0x560b83fba320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9df00033e0 tx=0x560b83fa6970 comp rx=0 tx=0).stop 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e00600640 1 -- 172.21.15.33:0/763563792 reap_dead start 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 shutdown_connections 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 >> 172.21.15.33:0/763563792 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 shutdown_connections 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 1 -- 172.21.15.33:0/763563792 wait complete. 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.013+0000 7f9e0324a480 20 client.4713 trim_cache size 0 max 16384 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.AlternateNameCreat (26 ms) 2024-03-28T01:08:24.069 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameSymlink 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.014+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.015+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.015+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.015+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.015+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b83fad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.015+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40075d0 con 0x560b83ee4360 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.015+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83fa9250 con 0x560b84053230 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.015+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f860 con 0x560b83eff1f0 2024-03-28T01:08:24.078 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b83fad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.079 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b83fad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60980/0 (socket says 172.21.15.33:60980) 2024-03-28T01:08:24.079 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 -- 172.21.15.33:0/675801587 learned_addr learned my addr 172.21.15.33:0/675801587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9dffc00640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9dff200640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 -- 172.21.15.33:0/675801587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 -- 172.21.15.33:0/675801587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 msgr2=0x560b83fba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 0x560b83fba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9e00600640 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83f29dd0 con 0x560b83ee4360 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.016+0000 7f9dff200640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b84059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e00600640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b83fad220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9df80014d0 tx=0x7f9df8060000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229e32f675cf8590 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:24.080 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df804c040 con 0x560b83ee4360 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df800c800 con 0x560b83ee4360 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 10 client.4716 Subscribing to map 'mdsmap' 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83f99a60 con 0x560b83ee4360 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9df6000640 20 client.4716 tick 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9df6000640 20 client.4716 collect_and_send_metrics 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9df6000640 20 client.4716 collect_and_send_global_metrics 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9df6000640 5 client.4716 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9df6000640 20 client.4716 trim_cache size 0 max 16384 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9df6000640 20 client.4716 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df806b030 con 0x560b83ee4360 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009070 con 0x560b83ee4360 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9dfca00640 1 client.4716 handle_mds_map epoch 8 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 20 client.4716 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:24.081 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 10 client.4716 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 20 client.4716 mds is 0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 10 client.4716 _open_mds_session mds.0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053da0 0x560b8404ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faa800 con 0x560b84053da0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.017+0000 7f9e0324a480 10 client.4716 waiting for session to mds.0 to open 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.018+0000 7f9dffc00640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053da0 0x560b8404ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.018+0000 7f9dffc00640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053da0 0x560b8404ede0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d7a14055da0814 server_cookie=81e00c25b0fd041a in_seq=0 out_seq=0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.018+0000 7f9dfca00640 10 client.4716 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84053da0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9dfca00640 10 client.4716 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9dfca00640 10 client.4716 renew_caps mds.0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400a9b0 con 0x560b84053da0 2024-03-28T01:08:24.082 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9dfca00640 10 client.4716 connect_mds_targets for mds.0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 10 client.4716 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 20 client.4716 mds is 0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 10 client.4716 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 20 client.4716 encode_cap_releases enter (req: 0x560b84053a70, mds: 0) 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 20 client.4716 send_request set sent_stamp to 2024-03-28T01:08:24.023092+0000 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 10 client.4716 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.018445+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.018445+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb4240 con 0x560b84053da0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9e0324a480 20 client.4716 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400a9b0 con 0x560b84053da0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.022+0000 7f9dfca00640 10 client.4716 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df00b5070 con 0x560b84053da0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 insert_trace from 2024-03-28T01:08:24.023092+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:24.083 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 features 0xffffffffffffffff 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 update_snap_trace len 70 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref now is 1 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 15 client.4716 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 15 client.4716 snapc 1=[] 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 2 -> 1 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 hrm is_target=1 is_dentry=0 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400acc0) - ctime 2024-03-28T01:08:23.997214+0000 mtime 2024-03-28T01:08:23.997214+0000 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 dir hash is 2 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 12 client.4716 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 22 2024-03-28T01:08:24.084 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 12 client.4716 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=- has_dir_layout 0x7f9dc400acc0) caps pAsLsXsFs 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref was 1 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref now is 2 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 15 client.4716 add_update_cap first one, opened snaprealm 0x7f9dc400abd0 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 2 -> 1 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 lat 0.000667 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 _ll_get 0x7f9dc400acc0 0x1 -> 1 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 3 client.4716 openat enter(AlternateNameSymlink_44790) 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.085 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 path_walk AlternateNameSymlink_44790 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 0 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) AlternateNameSymlink_44790 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 (path is AlternateNameSymlink_44790) 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 may_lookup 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 _getattr mask As issued=1 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 3 client.4716 may_lookup 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 _do_lookup on #0x1/AlternateNameSymlink_44790 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 choose_target_mds inode dir hash is 2 on AlternateNameSymlink_44790 => 1917129826 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 choose_target_mds 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=1917129826 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 mds is 0 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:24.086 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 encode_cap_releases enter (req: 0x560b83fb4240, mds: 0) 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 send_request set sent_stamp to 2024-03-28T01:08:24.023818+0000 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 send_request client_request(unknown.0:2 lookup #0x1/AlternateNameSymlink_44790 2024-03-28T01:08:24.023803+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/AlternateNameSymlink_44790 2024-03-28T01:08:24.023803+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb45a0 con 0x560b84053da0 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 awaiting reply|forward|kick on 0x7ffe68269390 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df0059030 con 0x560b84053da0 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 insert_trace from 2024-03-28T01:08:24.023818+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 10 client.4716 insert_trace -- no trace 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 handle_client_reply signalling caller 0x7ffe68269390 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 lat 0.000375 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 _do_lookup res is -2 2024-03-28T01:08:24.087 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 _lookup 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) AlternateNameSymlink_44790 = -2 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 path_walk cur=0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 path_walk 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 may_create 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 _getattr mask As issued=1 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 3 client.4716 may_create 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 8 client.4716 _create(0x1 AlternateNameSymlink_44790, 0777) 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 get_or_create 0x1.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) name AlternateNameSymlink_44790 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 15 open_dir 0x560b83fa9250 on 0x7f9dc400acc0 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 15 client.4716 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameSymlink_44790' dn 0x1.head["AlternateNameSymlink_44790"] (new dn) 2024-03-28T01:08:24.088 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 choose_target_mds inode dir hash is 2 on AlternateNameSymlink_44790 => 1917129826 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 choose_target_mds 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=1917129826 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 mds is 0 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 encode_cap_releases enter (req: 0x560b83fb45a0, mds: 0) 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 encode_dentry_release enter(dn:0x7f9dcc00a370) 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0), req:0x560b83fb45a0 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 send_request set sent_stamp to 2024-03-28T01:08:24.024266+0000 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 10 client.4716 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameSymlink_44790 (foo) 2024-03-28T01:08:24.024248+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameSymlink_44790 (foo) 2024-03-28T01:08:24.024248+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84056300 con 0x560b84053da0 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9e0324a480 20 client.4716 awaiting reply|forward|kick on 0x7ffe682696f0 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.023+0000 7f9dfca00640 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 850+0+0 (crc 0 0 0) 0x7f9df001ebf0 con 0x560b84053da0 2024-03-28T01:08:24.089 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 insert_trace from 2024-03-28T01:08:24.024266+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 features 0xffffffffffffffff 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 update_snap_trace len 70 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref was 1 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref now is 2 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 update_inode_file_time 0x1000000000c.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) - ctime 2024-03-28T01:08:24.024248+0000 mtime 2024-03-28T01:08:24.024248+0000 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 12 client.4716 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 12 client.4716 add_update_inode adding 0x1000000000c.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=- objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref was 2 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref now is 3 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 15 client.4716 add_update_cap first one, opened snaprealm 0x7f9dc400abd0 2024-03-28T01:08:24.090 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x1000000000c.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 12 client.4716 add_update_inode had 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) caps pAsLsXsFs 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 update_inode_file_time 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:23.997214+0000 ctime=2024-03-28T01:08:23.997214+0000 change_attr=22 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) pAsLsXs ctime 2024-03-28T01:08:24.024248+0000 mtime 2024-03-28T01:08:24.024248+0000 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 dir hash is 2 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 12 client.4716 add_update_inode client inode change_attr: 22 , mds inodestat change_attr: 23 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 10 client.4716 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 12 client.4716 insert_dentry_inode 'AlternateNameSymlink_44790' vino 0x1000000000c.head in dir 0x1.head dn 0x7f9dcc00a370 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 15 client.4716 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameSymlink_44790' to inode 0x7f9dc400b2a0 dn 0x1.head["AlternateNameSymlink_44790"] (old dn) 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 link inode 0x7f9dc400b2a0 parents now 0x1.head["AlternateNameSymlink_44790"] 2024-03-28T01:08:24.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 15 client.4716 update_dentry_lease 0x1.head["AlternateNameSymlink_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:24.024266+0000 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 3 -> 2 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 handle_client_reply signalling caller 0x7ffe682696f0 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9dfca00640 20 client.4716 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 client.4716 delegated_inos: [] 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 client.4716 make_request created ino 0x1000000000c 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 make_request target is 0x1000000000c.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 lat 0.004307 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x1000000000c.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x1000000000c.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 client.4716 _create_fh 0x1000000000c mode 2 2024-03-28T01:08:24.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 trim_cache size 1 max 16384 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 8 client.4716 create(#0x1/AlternateNameSymlink_44790, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 3 client.4716 openat exit(AlternateNameSymlink_44790) 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 client.4716 get_quota_root realm 0x1 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 client.4716 get_quota_root 0x1000000000c.head -> 0x1.head 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.027+0000 7f9e0324a480 10 client.4716 cur file size is 0 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9e0324a480 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f29150 0x560b8406a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- osd_op(unknown.0.0:1 3.22 3:46cca9ab:::1000000000c.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b8406a690 con 0x560b83f29150 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- osd_op(unknown.0.0:2 3.22 3:46cca9ab:::1000000000c.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f1d6b0 con 0x560b83f29150 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9dfca00640 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9dff200640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f29150 0x560b8406a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9dff200640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f29150 0x560b8406a150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=917075fe96e8cba server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.028+0000 7f9dfca00640 10 client.4716 ms_handle_connect on v2:172.21.15.33:6808/1101919847 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.029+0000 7f9dff200640 1 -- 172.21.15.33:0/675801587 <== osd.2 v2:172.21.15.33:6808/1101919847 1 ==== osd_op_reply(1 1000000000c.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de805d030 con 0x560b83f29150 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9dff200640 1 -- 172.21.15.33:0/675801587 <== osd.2 v2:172.21.15.33:6808/1101919847 2 ==== osd_op_reply(2 1000000000c.00000000 [create] v23'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de805d030 con 0x560b83f29150 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 wanted_max_size 0 -> 3 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 get_caps 0x1000000000c.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 clear_suid_sgid 0x1000000000c.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 5 client.4716 get_cap_ref got first FILE_BUFFER ref on 0x1000000000c.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 5 client.4716 get_cap_ref got first FILE_CACHE ref on 0x1000000000c.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000c.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) - -> Fw 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 cap_delay_requeue on 0x1000000000c.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 get_quota_root realm 0x1 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 get_quota_root 0x1000000000c.head -> 0x1.head 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 7 client.4716 wrote to 3, extending file size 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000c.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) Fw -> Fw 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 cap_delay_requeue on 0x1000000000c.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 3 client.4716 write(10, "...", 3, 0) = 3 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 3 client.4716 close enter(10) 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 8 client.4716 _release_fh 0x560b83fb4990 mode 2 on 0x1000000000c.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 _flush 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- osd_op(unknown.0.0:3 3.22 3:46cca9ab:::1000000000c.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f1d6b0 con 0x560b83f29150 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 check_caps on 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 delaying cap release 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 cap_delay_requeue on 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 _release_fh 0x560b83fb4990 on inode 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) no async_err state 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 3 client.4716 close exit(10) 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 path_walk cur=0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 path_walk 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 may_create 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 _getattr mask As issued=1 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 3 client.4716 may_create 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 8 client.4716 _symlink(0x1 AlternateNameSymlink_447902, AlternateNameSymlink_44790, uid 0, gid 0) 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 get_or_create 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) name AlternateNameSymlink_447902 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 15 client.4716 link dir 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameSymlink_447902' dn 0x1.head["AlternateNameSymlink_447902"] (new dn) 2024-03-28T01:08:24.096 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 choose_target_mds inode dir hash is 2 on AlternateNameSymlink_447902 => 3349540298 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 choose_target_mds 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=3349540298 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 choose_target_mds from dirfragtree hash 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 mds is 0 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 encode_cap_releases enter (req: 0x560b83f1d6b0, mds: 0) 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 encode_dentry_release enter(dn:0x560b83ea95b0) 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0), req:0x560b83f1d6b0 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 send_request set sent_stamp to 2024-03-28T01:08:24.031630+0000 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 10 client.4716 send_request client_request(unknown.0:4 symlink owner_uid=0, owner_gid=0 #0x1/AlternateNameSymlink_447902 (foo2) AlternateNameSymlink_44790 2024-03-28T01:08:24.031614+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 symlink owner_uid=0, owner_gid=0 #0x1/AlternateNameSymlink_447902 (foo2) AlternateNameSymlink_44790 2024-03-28T01:08:24.031614+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84056740 con 0x560b84053da0 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.030+0000 7f9e0324a480 20 client.4716 awaiting reply|forward|kick on 0x7ffe682698e0 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_reply(???:4 = 0 (0) Success unsafe) v1 ==== 860+0+0 (crc 0 0 0) 0x7f9df0063b70 con 0x560b84053da0 2024-03-28T01:08:24.097 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 handle_client_reply got a reply. Safe:0 tid 4 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 insert_trace from 2024-03-28T01:08:24.031630+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 features 0xffffffffffffffff 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 update_snap_trace len 70 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref was 2 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref now is 3 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 update_inode_file_time 0x1000000000d.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=120777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400b880) - ctime 2024-03-28T01:08:24.031614+0000 mtime 2024-03-28T01:08:24.031614+0000 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 size 0 -> 26 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode adding 0x1000000000d.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=- 0x7f9dc400b880) caps pAsLsXsFscr 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref was 3 2024-03-28T01:08:24.098 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref now is 4 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 15 client.4716 add_update_cap first one, opened snaprealm 0x7f9dc400abd0 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 add_update_cap issued - -> pAsLsXsFscr from mds.0 on 0x1000000000d.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode had 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) caps pAsLsXs 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 update_inode_file_time 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.024248+0000 ctime=2024-03-28T01:08:24.024248+0000 change_attr=23 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) pAsLsXs ctime 2024-03-28T01:08:24.031614+0000 mtime 2024-03-28T01:08:24.031614+0000 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 dir hash is 2 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode client inode change_attr: 23 , mds inodestat change_attr: 24 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 insert_dentry_inode 'AlternateNameSymlink_447902' vino 0x1000000000d.head in dir 0x1.head dn 0x560b83ea95b0 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 15 client.4716 link dir 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameSymlink_447902' to inode 0x7f9dc400b880 dn 0x1.head["AlternateNameSymlink_447902"] (old dn) 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 link inode 0x7f9dc400b880 parents now 0x1.head["AlternateNameSymlink_447902"] 2024-03-28T01:08:24.099 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 15 client.4716 update_dentry_lease 0x1.head["AlternateNameSymlink_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.031630+0000 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 4 -> 3 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 handle_client_reply signalling caller 0x7ffe682698e0 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 handle_client_reply awaiting kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 sendrecv kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 lat 0.001229 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 trim_cache size 2 max 16384 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 8 client.4716 _symlink("#0x1/AlternateNameSymlink_447902", "AlternateNameSymlink_44790") = 0 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 walk: AlternateNameSymlink_447902 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 path_walk cur=0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 path_walk AlternateNameSymlink_447902 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 0 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) AlternateNameSymlink_447902 2024-03-28T01:08:24.100 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 (path is AlternateNameSymlink_447902) 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 may_lookup 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 _getattr mask As issued=1 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 3 client.4716 may_lookup 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 _lookup have 0x1.head["AlternateNameSymlink_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 _do_lookup on #0x1/AlternateNameSymlink_447902 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 choose_target_mds inode dir hash is 2 on AlternateNameSymlink_447902 => 3349540298 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 choose_target_mds 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=3349540298 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 mds is 0 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 send_request rebuilding request 5 for mds.0 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 encode_cap_releases enter (req: 0x560b84056740, mds: 0) 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 send_request set sent_stamp to 2024-03-28T01:08:24.032939+0000 2024-03-28T01:08:24.101 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 10 client.4716 send_request client_request(unknown.0:5 lookup #0x1/AlternateNameSymlink_447902 2024-03-28T01:08:24.032915+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:5 lookup #0x1/AlternateNameSymlink_447902 2024-03-28T01:08:24.032915+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1dac0 con 0x560b84053da0 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9e0324a480 20 client.4716 awaiting reply|forward|kick on 0x7ffe68269650 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:5 = 0 (0) Success) v1 ==== 856+0+0 (crc 0 0 0) 0x7f9df006c840 con 0x560b84053da0 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 handle_client_reply got a reply. Safe:1 tid 5 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 insert_trace from 2024-03-28T01:08:24.032939+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 features 0xffffffffffffffff 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 update_snap_trace len 70 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref was 3 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 get_snap_realm 0x1 0x7f9dc400abd0, nref now is 4 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode had 0x1000000000d.head(faked_ino=0 nref=5 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) caps pAsLsXsFscr 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.102 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 add_update_cap issued pAsLsXsFscr -> pAsLsXsFscr from mds.0 on 0x1000000000d.head(faked_ino=0 nref=5 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode had 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) caps pAsLsXs 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 update_inode_file_time 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) pAsLsXs ctime 2024-03-28T01:08:24.031614+0000 mtime 2024-03-28T01:08:24.031614+0000 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 dir hash is 2 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 12 client.4716 add_update_inode client inode change_attr: 24 , mds inodestat change_attr: 24 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 10 client.4716 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.032+0000 7f9dfca00640 20 client.4716 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 12 client.4716 insert_dentry_inode 'AlternateNameSymlink_447902' vino 0x1000000000d.head in dir 0x1.head dn 0x560b83ea95b0 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 12 client.4716 had dentry AlternateNameSymlink_447902 with correct vino 0x1000000000d.head 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 15 client.4716 update_dentry_lease 0x1.head["AlternateNameSymlink_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.032939+0000 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 4 -> 3 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 handle_client_reply signalling caller 0x7ffe68269650 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 handle_client_reply awaiting kickback on tid 5 0x7f9dfc9fde80 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 20 client.4716 sendrecv kickback on tid 5 0x7f9dfc9fde80 2024-03-28T01:08:24.103 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 20 client.4716 make_request target is 0x1000000000d.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 20 client.4716 lat 0.000798 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 10 client.4716 _do_lookup res is 0 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 20 client.4716 _lookup have 0x1.head["AlternateNameSymlink_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dff200640 1 -- 172.21.15.33:0/675801587 <== osd.2 v2:172.21.15.33:6808/1101919847 3 ==== osd_op_reply(3 1000000000c.00000000 [write 0~3] v23'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de805d030 con 0x560b83f29150 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 10 client.4716 _lookup 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) AlternateNameSymlink_447902 = 0x1000000000d.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el9/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/test/client/alternate_name.cc:101: Failure 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stdout:Expected equality of these values: 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stdout: wdr.alternate_name 2024-03-28T01:08:24.104 INFO:tasks.workunit.client.0.smithi033.stdout: Which is: "" 2024-03-28T01:08:24.105 INFO:tasks.workunit.client.0.smithi033.stdout: altname2 2024-03-28T01:08:24.105 INFO:tasks.workunit.client.0.smithi033.stdout: Which is: "foo2" 2024-03-28T01:08:24.109 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 20 client.4716 symlink count 1, value is 'AlternateNameSymlink_44790' 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9df7400640 10 client.4716 _flushed 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9df7400640 5 client.4716 put_cap_ref dropped last FILE_BUFFER ref on 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9df7400640 5 client.4716 put_cap_ref dropped last FILE_CACHE ref on 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9df7400640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 2 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9df7400640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameSymlink_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 2 client.4716 unmount 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, size was 2+3 2024-03-28T01:08:24.110 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 trim_cache size 2 max 16384 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 15 client.4716 trim_dentry unlinking dn AlternateNameSymlink_44790 in dir 0x1 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 15 client.4716 unlink dir 0x7f9dc400acc0 'AlternateNameSymlink_44790' dn 0x7f9dcc00a370 inode 0x7f9dc400b2a0 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 unlink inode 0x7f9dc400b2a0 parents now 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 15 client.4716 unlink removing 'AlternateNameSymlink_44790' dn 0x7f9dcc00a370 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 2 client.4716 unmounting 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83faadc0 con 0x560b84053da0 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.033+0000 7f9e0324a480 10 client.4716 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_reply(???:4 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df000e000 con 0x560b84053da0 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 10 client.4716 insert_trace from 2024-03-28T01:08:24.031630+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 10 client.4716 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:24.111 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) parents=0x1.head["AlternateNameSymlink_447902"] 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, size was 1+3 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 trim_cache size 1 max 16384 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 15 client.4716 trim_dentry unlinking dn AlternateNameSymlink_447902 in dir 0x1 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 15 client.4716 unlink dir 0x7f9dc400acc0 'AlternateNameSymlink_447902' dn 0x560b83ea95b0 inode 0x7f9dc400b880 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 unlink inode 0x7f9dc400b880 parents now 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 15 client.4716 unlink removing 'AlternateNameSymlink_447902' dn 0x560b83ea95b0 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 15 client.4716 close_dir dir 0x560b83fa9250 on 0x7f9dc400acc0 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.112 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 _ll_drop_pins 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 _ll_put 0x7f9dc400acc0 0x1 1 -> 0 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 _release 0x1000000000d.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 _invalidate_inode_cache 0x1000000000d.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={1024=0} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 _flush 0x1000000000d.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 nothing to flush 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 _release 0x1000000000c.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 _invalidate_inode_cache 0x1000000000c.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 _flush 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 nothing to flush 2024-03-28T01:08:24.113 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 _release 0x1.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 _flush 0x1.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 nothing to flush 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000d.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) n = 1 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 20 client.4716 put_inode on 0x1.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 flush_caps_sync 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 check_caps on 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) wanted - used - issued pAsxLsXsxFsxcrwb revoking - flags=3 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 mark_caps_flushing Fw 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.114 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 mark_caps_clean 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 send_cap 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) mds.0 seq 1 used - want - flush Fw retain p held pAsxLsXsxFsxcrwb revoking - dropping AsxLsXsxFsxcrwb 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 15 client.4716 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x1000000000c 1 seq 1 tid 2 caps=p dirty=Fw wanted=- follows 1 size 3/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.031469+0000 ctime 2024-03-28T01:08:24.031469+0000 change_attr 1) v12 -- 0x560b83f98a50 con 0x560b84053da0 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 wait_sync_caps want 2 (last is 2, 1 total flushing) 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.037+0000 7f9e0324a480 10 client.4716 waiting on mds.0 tid 2 (want 2) 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_caps(flush_ack ino 0x1000000000c 1 seq 1 tid 2 caps=p dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x560b83f98a50 con 0x560b84053da0 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 5 client.4716 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 10 client.4716 mds.0 seq now 1 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 5 client.4716 handle_cap_flush_ack mds.0 cleaned Fw on 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=p(0=p) flushing_caps=Fw objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) with Fw 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 5 client.4716 flushing_caps Fw -> - 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 10 client.4716 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=p(0=p) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) !flushing 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 10 client.4716 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 20 client.4716 put_inode on 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=p(0=p) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, size was 0+3 2024-03-28T01:08:24.115 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 20 client.4716 trim_cache size 0 max 16384 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, size still 0+3 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.040+0000 7f9e0324a480 10 client.4716 wait_sync_caps want 2 (last is 2, 0 total flushing) 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 20 client.4716 trim_cache size 0 max 16384 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 _put_inode on 0x1.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 25 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 15 client.4716 remove_cap last one, closing snaprealm 0x7f9dc400abd0 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 3 -> 2 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=- has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.117 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 _put_inode on 0x1000000000c.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=p(0=p) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 14 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 remove_cap mds.0 on 0x1000000000c.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=p(0=p) objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 15 client.4716 remove_cap last one, closing snaprealm 0x7f9dc400abd0 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 2 -> 1 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 _put_inode deleting 0x1000000000c.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.024248+0000 mtime=2024-03-28T01:08:24.031469+0000 ctime=2024-03-28T01:08:24.031469+0000 change_attr=1 caps=- objectset[0x1000000000c ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 _put_inode on 0x1000000000d.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={1024=0} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) n = 10 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 remove_cap mds.0 on 0x1000000000d.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=pAsLsXsFscr(0=pAsLsXsFscr) 0x7f9dc400b880) 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 15 client.4716 remove_cap last one, closing snaprealm 0x7f9dc400abd0 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 20 client.4716 put_snap_realm 0x1 0x7f9dc400abd0 1 -> 0 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 10 client.4716 _put_inode deleting 0x1000000000d.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=120777 size=26/0 nlink=1 btime=2024-03-28T01:08:24.031614+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=0 caps=- 0x7f9dc400b880) 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 2 client.4716 _close_mds_session mds.0 seq 1 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 1) v7 -- 0x560b83f2b790 con 0x560b84053da0 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.041+0000 7f9e0324a480 2 client.4716 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:24.118 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84053da0 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 10 client.4716 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 5 client.4716 _closed_mds_session mds.0 seq 1 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 1 -- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053da0 msgr2=0x560b8404ede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053da0 0x560b8404ede0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 10 client.4716 remove_session_caps mds.0 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 10 client.4716 kick_requests_closed for mds.0 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, size was 0+0 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 20 client.4716 trim_cache size 0 max 16384 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9dfca00640 10 client.4716 unmounting: trim pass, size still 0+0 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9e0324a480 2 client.4716 unmounted. 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9e0324a480 1 client.4716 shutdown 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f29150 msgr2=0x560b8406a150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.119 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.044+0000 7f9e0324a480 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f29150 0x560b8406a150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 msgr2=0x560b83fad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e0324a480 1 --2- 172.21.15.33:0/675801587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b83fad220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9df80014d0 tx=0x7f9df8060000 comp rx=0 tx=0).stop 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e00600640 1 -- 172.21.15.33:0/675801587 reap_dead start 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 shutdown_connections 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 >> 172.21.15.33:0/675801587 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 shutdown_connections 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e0324a480 1 -- 172.21.15.33:0/675801587 wait complete. 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.045+0000 7f9e0324a480 20 client.4716 trim_cache size 0 max 16384 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stdout:[ FAILED ] TestClient.AlternateNameSymlink (32 ms) 2024-03-28T01:08:24.120 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameRename 2024-03-28T01:08:24.130 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.046+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.046+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 0x560b83fad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4005550 con 0x560b83ee4360 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f295d0 con 0x560b84053230 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8404f970 con 0x560b83eff1f0 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e00600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60982/0 (socket says 172.21.15.33:60982) 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9e00600640 1 -- 172.21.15.33:0/184701653 learned_addr learned my addr 172.21.15.33:0/184701653 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9dffc00640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 0x560b83fad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.047+0000 7f9dff200640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.131 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e00600640 1 -- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 msgr2=0x560b83fba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e00600640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83eff1f0 0x560b83fba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e00600640 1 -- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 msgr2=0x560b83fad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e00600640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b84053230 0x560b83fad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e00600640 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b84055130 con 0x560b83ee4360 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e00600640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b84059ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000ff0 tx=0x7f9df801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca70108fdcea91d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df804c040 con 0x560b83ee4360 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df80036b0 con 0x560b83ee4360 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e0324a480 10 client.4719 Subscribing to map 'mdsmap' 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.048+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83fbab90 con 0x560b83ee4360 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9df5600640 20 client.4719 tick 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9df5600640 20 client.4719 collect_and_send_metrics 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9df5600640 20 client.4719 collect_and_send_global_metrics 2024-03-28T01:08:24.132 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9df5600640 5 client.4719 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9df5600640 20 client.4719 trim_cache size 0 max 16384 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9df5600640 20 client.4719 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8073060 con 0x560b83ee4360 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df8009070 con 0x560b83ee4360 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9dfde00640 1 client.4719 handle_mds_map epoch 8 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9e0324a480 20 client.4719 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9e0324a480 10 client.4719 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9e0324a480 20 client.4719 mds is 0 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9e0324a480 10 client.4719 _open_mds_session mds.0 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9e0324a480 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056490 0x560b83fb4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84056490 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9e0324a480 10 client.4719 waiting for session to mds.0 to open 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.049+0000 7f9dffc00640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056490 0x560b83fb4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.050+0000 7f9dfde00640 10 client.4719 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:24.133 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.050+0000 7f9dffc00640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056490 0x560b83fb4110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8bf7d96c4ba6e8 server_cookie=5287419091a74085 in_seq=0 out_seq=0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84056490 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9dfde00640 10 client.4719 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9dfde00640 10 client.4719 renew_caps mds.0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc4002f70 con 0x560b84056490 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9dfde00640 10 client.4719 connect_mds_targets for mds.0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 10 client.4719 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 20 client.4719 mds is 0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 10 client.4719 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 20 client.4719 encode_cap_releases enter (req: 0x560b83f1d6b0, mds: 0) 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 20 client.4719 send_request set sent_stamp to 2024-03-28T01:08:24.054358+0000 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 10 client.4719 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.050030+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.050030+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2b610 con 0x560b84056490 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9e0324a480 20 client.4719 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:24.134 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc4002f70 con 0x560b84056490 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.053+0000 7f9dfde00640 10 client.4719 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059040 con 0x560b84056490 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 insert_trace from 2024-03-28T01:08:24.054358+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 features 0xffffffffffffffff 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 update_snap_trace len 70 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref now is 1 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 15 client.4719 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 15 client.4719 snapc 1=[] 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.135 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 put_snap_realm 0x1 0x7f9dc4003190 2 -> 1 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 hrm is_target=1 is_dentry=0 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400acc0) - ctime 2024-03-28T01:08:24.031614+0000 mtime 2024-03-28T01:08:24.031614+0000 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 dir hash is 2 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 12 client.4719 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 24 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 12 client.4719 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=- has_dir_layout 0x7f9dc400acc0) caps pAsLsXsFs 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref was 1 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref now is 2 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 15 client.4719 add_update_cap first one, opened snaprealm 0x7f9dc4003190 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 put_snap_realm 0x1 0x7f9dc4003190 2 -> 1 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:24.136 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 lat 0.000555 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 _ll_get 0x7f9dc400acc0 0x1 -> 1 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 3 client.4719 openat enter(AlternateNameRename_44790) 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 path_walk AlternateNameRename_44790 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 0 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) AlternateNameRename_44790 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 (path is AlternateNameRename_44790) 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 may_lookup 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 _getattr mask As issued=1 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 3 client.4719 may_lookup 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.137 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 _do_lookup on #0x1/AlternateNameRename_44790 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 choose_target_mds inode dir hash is 2 on AlternateNameRename_44790 => 4023868197 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 choose_target_mds 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=4023868197 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 mds is 0 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 encode_cap_releases enter (req: 0x560b83f2b610, mds: 0) 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 send_request set sent_stamp to 2024-03-28T01:08:24.054985+0000 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 send_request client_request(unknown.0:2 lookup #0x1/AlternateNameRename_44790 2024-03-28T01:08:24.054969+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/AlternateNameRename_44790 2024-03-28T01:08:24.054969+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f2ca90 con 0x560b84056490 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 awaiting reply|forward|kick on 0x7ffe68269390 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df0005a80 con 0x560b84056490 2024-03-28T01:08:24.138 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 insert_trace from 2024-03-28T01:08:24.054985+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 10 client.4719 insert_trace -- no trace 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 handle_client_reply signalling caller 0x7ffe68269390 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 handle_client_reply awaiting kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 sendrecv kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 lat 0.000326 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 _do_lookup res is -2 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 _lookup 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) AlternateNameRename_44790 = -2 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 path_walk cur=0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 path_walk 2024-03-28T01:08:24.139 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 may_create 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 _getattr mask As issued=1 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 3 client.4719 may_create 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 8 client.4719 _create(0x1 AlternateNameRename_44790, 0777) 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 get_or_create 0x1.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) name AlternateNameRename_44790 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 15 open_dir 0x560b8406bdf0 on 0x7f9dc400acc0 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 15 client.4719 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameRename_44790' dn 0x1.head["AlternateNameRename_44790"] (new dn) 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 choose_target_mds inode dir hash is 2 on AlternateNameRename_44790 => 4023868197 2024-03-28T01:08:24.140 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 choose_target_mds 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=4023868197 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 mds is 0 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 encode_cap_releases enter (req: 0x560b83f2ca90, mds: 0) 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 encode_dentry_release enter(dn:0x7f9dc400a8c0) 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0), req:0x560b83f2ca90 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 send_request set sent_stamp to 2024-03-28T01:08:24.055382+0000 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 10 client.4719 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRename_44790 (foo) 2024-03-28T01:08:24.055363+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRename_44790 (foo) 2024-03-28T01:08:24.055363+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84053a70 con 0x560b84056490 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9e0324a480 20 client.4719 awaiting reply|forward|kick on 0x7ffe682696f0 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.054+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 849+0+0 (crc 0 0 0) 0x7f9df004adc0 con 0x560b84056490 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 insert_trace from 2024-03-28T01:08:24.055382+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 features 0xffffffffffffffff 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 update_snap_trace len 70 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref was 1 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref now is 2 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1000000000e.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) - ctime 2024-03-28T01:08:24.055363+0000 mtime 2024-03-28T01:08:24.055363+0000 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 12 client.4719 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 12 client.4719 add_update_inode adding 0x1000000000e.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=- objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref was 2 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref now is 3 2024-03-28T01:08:24.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 15 client.4719 add_update_cap first one, opened snaprealm 0x7f9dc4003190 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x1000000000e.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 12 client.4719 add_update_inode had 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) caps pAsLsXsFs 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.031614+0000 ctime=2024-03-28T01:08:24.031614+0000 change_attr=24 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) pAsLsXs ctime 2024-03-28T01:08:24.055363+0000 mtime 2024-03-28T01:08:24.055363+0000 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 dir hash is 2 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 12 client.4719 add_update_inode client inode change_attr: 24 , mds inodestat change_attr: 25 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 10 client.4719 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 12 client.4719 insert_dentry_inode 'AlternateNameRename_44790' vino 0x1000000000e.head in dir 0x1.head dn 0x7f9dc400a8c0 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 15 client.4719 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameRename_44790' to inode 0x7f9dc400b2a0 dn 0x1.head["AlternateNameRename_44790"] (old dn) 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 link inode 0x7f9dc400b2a0 parents now 0x1.head["AlternateNameRename_44790"] 2024-03-28T01:08:24.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 15 client.4719 update_dentry_lease 0x1.head["AlternateNameRename_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:24.055382+0000 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 put_snap_realm 0x1 0x7f9dc4003190 3 -> 2 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 handle_client_reply signalling caller 0x7ffe682696f0 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 handle_client_reply awaiting kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 sendrecv kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 client.4719 delegated_inos: [] 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 client.4719 make_request created ino 0x1000000000e 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 make_request target is 0x1000000000e.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 lat 0.005049 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x1000000000e.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x1000000000e.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 client.4719 _create_fh 0x1000000000e mode 2 2024-03-28T01:08:24.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 trim_cache size 1 max 16384 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 8 client.4719 create(#0x1/AlternateNameRename_44790, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 3 client.4719 openat exit(AlternateNameRename_44790) 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 client.4719 get_quota_root realm 0x1 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 client.4719 get_quota_root 0x1000000000e.head -> 0x1.head 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 10 client.4719 cur file size is 0 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f29150 0x560b83f295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:1 3.37 3:eefe613f:::1000000000e.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b8404e4d0 con 0x560b83f29150 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:2 3.37 3:eefe613f:::1000000000e.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f1dd40 con 0x560b83f29150 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.059+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.060+0000 7f9dff200640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f29150 0x560b83f295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.061+0000 7f9dff200640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f29150 0x560b83f295b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=e447fe3e53ecf437 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.061+0000 7f9dfde00640 10 client.4719 ms_handle_connect on v2:172.21.15.194:6824/2849955959 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.062+0000 7f9dff200640 1 -- 172.21.15.33:0/184701653 <== osd.7 v2:172.21.15.194:6824/2849955959 1 ==== osd_op_reply(1 1000000000e.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de805d030 con 0x560b83f29150 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9dff200640 1 -- 172.21.15.33:0/184701653 <== osd.7 v2:172.21.15.194:6824/2849955959 2 ==== osd_op_reply(2 1000000000e.00000000 [create] v23'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de805d030 con 0x560b83f29150 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 wanted_max_size 0 -> 3 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 get_caps 0x1000000000e.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 clear_suid_sgid 0x1000000000e.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 5 client.4719 get_cap_ref got first FILE_BUFFER ref on 0x1000000000e.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 5 client.4719 get_cap_ref got first FILE_CACHE ref on 0x1000000000e.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000e.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) - -> Fw 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 cap_delay_requeue on 0x1000000000e.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 get_quota_root realm 0x1 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 get_quota_root 0x1000000000e.head -> 0x1.head 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 7 client.4719 wrote to 3, extending file size 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000e.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) Fw -> Fw 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 cap_delay_requeue on 0x1000000000e.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 3 client.4719 write(10, "...", 3, 0) = 3 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 3 client.4719 close enter(10) 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 8 client.4719 _release_fh 0x560b84056990 mode 2 on 0x1000000000e.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 _flush 0x1000000000e.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:3 3.37 3:eefe613f:::1000000000e.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b8404e4d0 con 0x560b83f29150 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 check_caps on 0x1000000000e.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 delaying cap release 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 cap_delay_requeue on 0x1000000000e.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.147 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 _release_fh 0x560b84056990 on inode 0x1000000000e.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) no async_err state 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 3 client.4719 close exit(10) 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 path_walk cur=0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 path_walk 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 path_walk cur=0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 path_walk 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 may_delete 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0); ; name AlternateNameRename_44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 _getattr mask As issued=1 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 3 client.4719 may_delete 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 may_delete 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0); ; name AlternateNameRename_447902; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 _getattr mask As issued=1 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 3 client.4719 may_delete 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 8 client.4719 _rename(0x1 AlternateNameRename_44790 to 0x1 AlternateNameRename_447902 uid 0 gid 0) 2024-03-28T01:08:24.148 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 get_or_create 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) name AlternateNameRename_44790 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 get_or_create 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) name AlternateNameRename_447902 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 15 client.4719 link dir 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameRename_447902' dn 0x1.head["AlternateNameRename_447902"] (new dn) 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 _lookup have 0x1.head["AlternateNameRename_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 _lookup 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) AlternateNameRename_44790 = 0x1000000000e.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x1000000000e.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x1000000000e.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 _lookup have 0x1.head["AlternateNameRename_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 _do_lookup on #0x1/AlternateNameRename_447902 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 choose_target_mds inode dir hash is 2 on AlternateNameRename_447902 => 1034923473 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 choose_target_mds 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=1034923473 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 mds is 0 2024-03-28T01:08:24.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 encode_cap_releases enter (req: 0x560b840540b0, mds: 0) 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 send_request set sent_stamp to 2024-03-28T01:08:24.066582+0000 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 10 client.4719 send_request client_request(unknown.0:4 lookup #0x1/AlternateNameRename_447902 2024-03-28T01:08:24.066568+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 lookup #0x1/AlternateNameRename_447902 2024-03-28T01:08:24.066568+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1e150 con 0x560b84056490 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.065+0000 7f9e0324a480 20 client.4719 awaiting reply|forward|kick on 0x7ffe68269600 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_reply(???:4 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df00637b0 con 0x560b84056490 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 20 client.4719 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 insert_trace from 2024-03-28T01:08:24.066582+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 insert_trace -- no trace 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 20 client.4719 handle_client_reply signalling caller 0x7ffe68269600 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 20 client.4719 handle_client_reply awaiting kickback on tid 4 0x7f9dfddfde80 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 sendrecv kickback on tid 4 0x7f9dfddfde80 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 lat 0.000409 2024-03-28T01:08:24.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 10 client.4719 _do_lookup res is -2 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 10 client.4719 _lookup 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) AlternateNameRename_447902 = -2 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 choose_target_mds inode dir hash is 2 on AlternateNameRename_447902 => 1034923473 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 choose_target_mds 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=1034923473 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 10 client.4719 choose_target_mds from dirfragtree hash 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 mds is 0 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 10 client.4719 send_request rebuilding request 5 for mds.0 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 encode_cap_releases enter (req: 0x560b8404e4d0, mds: 0) 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 encode_inode_release enter(in:0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0), req:0x560b8404e4d0 mds:0, drop:Ls, unless:-, force:0) 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 encode_dentry_release enter(dn:0x560b83e28110) 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0), req:0x560b8404e4d0 mds:0, drop:XxFs, unless:Fx, force:1) 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 encode_dentry_release enter(dn:0x7f9dc400a8c0) 2024-03-28T01:08:24.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0), req:0x560b8404e4d0 mds:0, drop:LxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 send_request set sent_stamp to 2024-03-28T01:08:24.067043+0000 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 10 client.4719 send_request client_request(unknown.0:5 rename #0x1/AlternateNameRename_447902 (foo2) #0x1/AlternateNameRename_44790 2024-03-28T01:08:24.067009+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:5 rename #0x1/AlternateNameRename_447902 (foo2) #0x1/AlternateNameRename_44790 2024-03-28T01:08:24.067009+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1e150 con 0x560b84056490 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9e0324a480 20 client.4719 awaiting reply|forward|kick on 0x7ffe68269850 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_caps(revoke ino 0x1000000000e 1 seq 2 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.055363+0000 ctime 2024-03-28T01:08:24.055363+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081070 con 0x560b84056490 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 5 client.4719 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 mds.0 seq now 1 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 5 client.4719 handle_cap_grant on in 0x1000000000e mds.0 seq 2 caps now pAsXsFsxcrwb was pAsxXsxFsxcrwb 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) pAsxXsxFsxcrwb ctime 2024-03-28T01:08:24.055363+0000 mtime 2024-03-28T01:08:24.055363+0000 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 revocation of AxXx 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 check_caps on 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) wanted Fxcb used Fcb issued pAsXsFsxcrwb revoking AxXx flags=1 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsxFsxcrwb revoking AxXx 2024-03-28T01:08:24.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 completed revocation of AxXx 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 mark_caps_flushing Fw 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 mark_caps_clean 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 send_cap 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) mds.0 seq 2 used Fcb want Fxcb flush Fw retain pAsLsxXsFsxcrwbl held pAsxXsxFsxcrwb revoking AxXx dropping - 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 15 client.4719 auth cap, requesting max_size 3 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x1000000000e 1 seq 2 tid 2 caps=pAsXsFsxcrwb dirty=Fw wanted=Fxcb follows 1 size 3/3 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.066436+0000 ctime 2024-03-28T01:08:24.066436+0000 change_attr 1) v12 -- 0x7f9dc400b880 con 0x560b84056490 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.066+0000 7f9dfde00640 10 client.4719 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_reply(???:5 = 0 (0) Success unsafe) v1 ==== 829+0+0 (crc 0 0 0) 0x7f9df00639a0 con 0x560b84056490 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 handle_client_reply got a reply. Safe:0 tid 5 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 insert_trace from 2024-03-28T01:08:24.067043+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 features 0xffffffffffffffff 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 update_snap_trace len 70 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref was 2 2024-03-28T01:08:24.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref now is 3 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 12 client.4719 add_update_inode had 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) caps pAsLsXsFsxcrwb 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.066436+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) pAsXsFsxcrwb ctime 2024-03-28T01:08:24.067009+0000 mtime 2024-03-28T01:08:24.055363+0000 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 12 client.4719 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 1 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 add_update_cap issued pAsXsFsxcrwb -> pAsLsXsFsxcrwb from mds.0 on 0x1000000000e.head(faked_ino=0 nref=14 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_44790"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 12 client.4719 add_update_inode had 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) caps pAsLsXs 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.055363+0000 ctime=2024-03-28T01:08:24.055363+0000 change_attr=25 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) pAsLsXs ctime 2024-03-28T01:08:24.067009+0000 mtime 2024-03-28T01:08:24.067009+0000 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 dir hash is 2 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 12 client.4719 add_update_inode client inode change_attr: 25 , mds inodestat change_attr: 27 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 10 client.4719 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 12 client.4719 insert_dentry_inode 'AlternateNameRename_447902' vino 0x1000000000e.head in dir 0x1.head dn 0x560b83e28110 2024-03-28T01:08:24.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 15 client.4719 unlink dir 0x7f9dc400acc0 'AlternateNameRename_44790' dn 0x7f9dc400a8c0 inode 0x7f9dc400b2a0 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 unlink inode 0x7f9dc400b2a0 parents now 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 15 client.4719 unlink removing 'AlternateNameRename_44790' dn 0x7f9dc400a8c0 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 15 client.4719 link dir 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 'AlternateNameRename_447902' to inode 0x7f9dc400b2a0 dn 0x1.head["AlternateNameRename_447902"] (old dn) 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 link inode 0x7f9dc400b2a0 parents now 0x1.head["AlternateNameRename_447902"] 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 15 client.4719 update_dentry_lease 0x1.head["AlternateNameRename_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.067043+0000 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 put_snap_realm 0x1 0x7f9dc4003190 3 -> 2 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 handle_client_reply signalling caller 0x7ffe68269850 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9dfde00640 20 client.4719 handle_client_reply awaiting kickback on tid 5 0x7f9dfddfde80 2024-03-28T01:08:24.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 sendrecv kickback on tid 5 0x7f9dfddfde80 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 make_request target is 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 lat 0.001551 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 10 client.4719 rename result is 0 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 trim_cache size 1 max 16384 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 8 client.4719 _rename(#0x1/AlternateNameRename_44790, #0x1/AlternateNameRename_447902) = 0 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 10 client.4719 walk: AlternateNameRename_447902 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 path_walk cur=0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 10 client.4719 path_walk AlternateNameRename_447902 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 10 client.4719 0 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) AlternateNameRename_447902 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 (path is AlternateNameRename_447902) 2024-03-28T01:08:24.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 may_lookup 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 10 client.4719 _getattr mask As issued=1 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 3 client.4719 may_lookup 0x7f9dc400acc0 = 0 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 _lookup have 0x1.head["AlternateNameRename_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 10 client.4719 _do_lookup on #0x1/AlternateNameRename_447902 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.067+0000 7f9e0324a480 20 client.4719 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 20 client.4719 choose_target_mds inode dir hash is 2 on AlternateNameRename_447902 => 1034923473 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 20 client.4719 choose_target_mds 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) is_hash=1 hash=1034923473 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 10 client.4719 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 20 client.4719 mds is 0 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 10 client.4719 send_request rebuilding request 6 for mds.0 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 20 client.4719 encode_cap_releases enter (req: 0x560b83f1e150, mds: 0) 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 20 client.4719 send_request set sent_stamp to 2024-03-28T01:08:24.068695+0000 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 10 client.4719 send_request client_request(unknown.0:6 lookup #0x1/AlternateNameRename_447902 2024-03-28T01:08:24.068655+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.157 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:6 lookup #0x1/AlternateNameRename_447902 2024-03-28T01:08:24.068655+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1e590 con 0x560b84056490 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.068+0000 7f9e0324a480 20 client.4719 awaiting reply|forward|kick on 0x7ffe68269650 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.069+0000 7f9dff200640 1 -- 172.21.15.33:0/184701653 <== osd.7 v2:172.21.15.194:6824/2849955959 3 ==== osd_op_reply(3 1000000000e.00000000 [write 0~3] v23'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9de805d030 con 0x560b83f29150 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.069+0000 7f9df7400640 10 client.4719 _flushed 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.069+0000 7f9df7400640 5 client.4719 put_cap_ref dropped last FILE_BUFFER ref on 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.069+0000 7f9df7400640 5 client.4719 put_cap_ref dropped last FILE_CACHE ref on 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.069+0000 7f9df7400640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 2 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.069+0000 7f9df7400640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:6 = 0 (0) Success) v1 ==== 829+0+0 (crc 0 0 0) 0x7f9df006c480 con 0x560b84056490 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 handle_client_reply got a reply. Safe:1 tid 6 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 insert_trace from 2024-03-28T01:08:24.068695+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 features 0xffffffffffffffff 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 update_snap_trace len 70 2024-03-28T01:08:24.158 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref was 2 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 get_snap_realm 0x1 0x7f9dc4003190, nref now is 3 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 12 client.4719 add_update_inode had 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) caps pAsLsXsFsxcrwb 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) pAsLsXsFsxcrwb ctime 2024-03-28T01:08:24.067009+0000 mtime 2024-03-28T01:08:24.066436+0000 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 12 client.4719 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 1 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 add_update_cap issued pAsLsXsFsxcrwb -> pAsLsXsFsxcrwb from mds.0 on 0x1000000000e.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 12 client.4719 add_update_inode had 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) caps pAsLsXsFs 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 update_inode_file_time 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400acc0) pAsLsXs ctime 2024-03-28T01:08:24.067009+0000 mtime 2024-03-28T01:08:24.067009+0000 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 dir hash is 2 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 12 client.4719 add_update_inode client inode change_attr: 27 , mds inodestat change_attr: 27 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.159 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 12 client.4719 insert_dentry_inode 'AlternateNameRename_447902' vino 0x1000000000e.head in dir 0x1.head dn 0x560b83e28110 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 12 client.4719 had dentry AlternateNameRename_447902 with correct vino 0x1000000000e.head 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 15 client.4719 update_dentry_lease 0x1.head["AlternateNameRename_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.068695+0000 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_snap_realm 0x1 0x7f9dc4003190 3 -> 2 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 handle_client_reply signalling caller 0x7ffe68269650 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 handle_client_reply awaiting kickback on tid 6 0x7f9dfddfde80 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 sendrecv kickback on tid 6 0x7f9dfddfde80 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 make_request target is 0x1000000000e.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 lat 0.004735 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 _do_lookup res is 0 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 _lookup have 0x1.head["AlternateNameRename_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 _lookup 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) AlternateNameRename_447902 = 0x1000000000e.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) 2024-03-28T01:08:24.160 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 2 client.4719 unmount 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, size was 1+2 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 trim_cache size 1 max 16384 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, size still 1+2 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 2 client.4719 unmounting 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83faa800 con 0x560b84056490 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_reply(???:5 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df006c770 con 0x560b84056490 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 waiting for write request '5' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 handle_client_reply got a reply. Safe:1 tid 5 2024-03-28T01:08:24.161 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 insert_trace from 2024-03-28T01:08:24.067043+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRename_447902"] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, size was 1+2 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 trim_cache size 1 max 16384 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 15 client.4719 trim_dentry unlinking dn AlternateNameRename_447902 in dir 0x1 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 15 client.4719 unlink dir 0x7f9dc400acc0 'AlternateNameRename_447902' dn 0x560b83e28110 inode 0x7f9dc400b2a0 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 unlink inode 0x7f9dc400b2a0 parents now 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 15 client.4719 unlink removing 'AlternateNameRename_447902' dn 0x560b83e28110 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 15 client.4719 close_dir dir 0x560b8406bdf0 on 0x7f9dc400acc0 2024-03-28T01:08:24.162 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_caps(flush_ack ino 0x1000000000e 1 seq 2 tid 2 caps=pAsXsFsxcrwb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400b880 con 0x560b84056490 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 _ll_drop_pins 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 _ll_put 0x7f9dc400acc0 0x1 1 -> 0 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 _release 0x1000000000e.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 _invalidate_inode_cache 0x1000000000e.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 _flush 0x1000000000e.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 nothing to flush 2024-03-28T01:08:24.163 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 _release 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 _flush 0x1.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 nothing to flush 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 20 client.4719 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 1 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 flush_caps_sync 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 wait_sync_caps want 2 (last is 2, 1 total flushing) 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.072+0000 7f9e0324a480 10 client.4719 waiting on mds.0 tid 2 (want 2) 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 10 client.4719 mds.0 seq now 2 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 5 client.4719 handle_cap_flush_ack mds.0 cleaned Fw on 0x1000000000e.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) with Fw 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 5 client.4719 flushing_caps Fw -> - 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 10 client.4719 0x1000000000e.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) !flushing 2024-03-28T01:08:24.164 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 10 client.4719 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 20 client.4719 put_inode on 0x1000000000e.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 1 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, size was 0+2 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 20 client.4719 trim_cache size 0 max 16384 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, size still 0+2 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 10 client.4719 wait_sync_caps want 2 (last is 2, 0 total flushing) 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 20 client.4719 trim_cache size 0 max 16384 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 10 client.4719 _put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) n = 24 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 10 client.4719 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 15 client.4719 remove_cap last one, closing snaprealm 0x7f9dc4003190 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 20 client.4719 put_snap_realm 0x1 0x7f9dc4003190 2 -> 1 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 10 client.4719 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=- has_dir_layout 0x7f9dc400acc0) 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 10 client.4719 _put_inode on 0x1000000000e.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) n = 26 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 10 client.4719 remove_cap mds.0 on 0x1000000000e.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.165 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 15 client.4719 remove_cap last one, closing snaprealm 0x7f9dc4003190 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 20 client.4719 put_snap_realm 0x1 0x7f9dc4003190 1 -> 0 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 10 client.4719 _put_inode deleting 0x1000000000e.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.055363+0000 mtime=2024-03-28T01:08:24.066436+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=1 caps=- objectset[0x1000000000e ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b2a0) 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 2 client.4719 _close_mds_session mds.0 seq 2 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 2) v7 -- 0x560b8404ea40 con 0x560b84056490 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.073+0000 7f9e0324a480 2 client.4719 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.076+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 <== mds.0 v2:172.21.15.33:6834/2290842450 12 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b84056490 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.076+0000 7f9dfde00640 10 client.4719 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.076+0000 7f9dfde00640 5 client.4719 _closed_mds_session mds.0 seq 2 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.076+0000 7f9dfde00640 1 -- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056490 msgr2=0x560b83fb4110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.076+0000 7f9dfde00640 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84056490 0x560b83fb4110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9dfde00640 10 client.4719 remove_session_caps mds.0 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9dfde00640 10 client.4719 kick_requests_closed for mds.0 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, size was 0+0 2024-03-28T01:08:24.166 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9dfde00640 20 client.4719 trim_cache size 0 max 16384 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9dfde00640 10 client.4719 unmounting: trim pass, size still 0+0 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 2 client.4719 unmounted. 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 client.4719 shutdown 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f29150 msgr2=0x560b83f295b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f29150 0x560b83f295b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 msgr2=0x560b84059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 --2- 172.21.15.33:0/184701653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83ee4360 0x560b84059ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000ff0 tx=0x7f9df801c000 comp rx=0 tx=0).stop 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e00600640 1 -- 172.21.15.33:0/184701653 reap_dead start 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 shutdown_connections 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 >> 172.21.15.33:0/184701653 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 shutdown_connections 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 1 -- 172.21.15.33:0/184701653 wait complete. 2024-03-28T01:08:24.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.077+0000 7f9e0324a480 20 client.4719 trim_cache size 0 max 16384 2024-03-28T01:08:24.168 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.AlternateNameRename (32 ms) 2024-03-28T01:08:24.168 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameRenameExistMatch 2024-03-28T01:08:24.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.078+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:24.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.078+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:24.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8406ccb0 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84053230 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8406bdf0 0x560b83f2ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4007d60 con 0x560b8406ccb0 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2e5f0 con 0x560b8406bdf0 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbb540 con 0x560b84053230 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8406ccb0 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8406ccb0 0x560b83fba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60986/0 (socket says 172.21.15.33:60986) 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9dffc00640 1 -- 172.21.15.33:0/3250881336 learned_addr learned my addr 172.21.15.33:0/3250881336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9e00600640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8406bdf0 0x560b83f2ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.079+0000 7f9dff200640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84053230 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dffc00640 1 -- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84053230 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b84053230 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dffc00640 1 -- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8406bdf0 msgr2=0x560b83f2ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b8406bdf0 0x560b83f2ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dffc00640 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8406b050 con 0x560b8406ccb0 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8406ccb0 0x560b83fba320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9df0017f80 tx=0x7f9df00ac000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e74ec3882b782a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5040 con 0x560b8406ccb0 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df0063da0 con 0x560b8406ccb0 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9e0324a480 10 client.4722 Subscribing to map 'mdsmap' 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b84061920 con 0x560b8406ccb0 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9df6000640 20 client.4722 tick 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9df6000640 20 client.4722 collect_and_send_metrics 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9df6000640 20 client.4722 collect_and_send_global_metrics 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9df6000640 5 client.4722 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9df6000640 20 client.4722 trim_cache size 0 max 16384 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.080+0000 7f9df6000640 20 client.4722 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:24.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b83f2a190 con 0x560b8406ccb0 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df00a1040 con 0x560b8406ccb0 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9dfca00640 1 client.4722 handle_mds_map epoch 8 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9e0324a480 20 client.4722 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9e0324a480 10 client.4722 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9e0324a480 10 client.4722 _open_mds_session mds.0 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83ee4360 0x560b83fb4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b83ee4360 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9e0324a480 10 client.4722 waiting for session to mds.0 to open 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9dff200640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83ee4360 0x560b83fb4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9dff200640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83ee4360 0x560b83fb4070 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e99f3179fe7fb2b server_cookie=62ca612b1e3ac502 in_seq=0 out_seq=0 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.081+0000 7f9dfca00640 10 client.4722 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b83ee4360 2024-03-28T01:08:24.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9dfca00640 10 client.4722 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9dfca00640 10 client.4722 renew_caps mds.0 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400acc0 con 0x560b83ee4360 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9dfca00640 10 client.4722 connect_mds_targets for mds.0 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 10 client.4722 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b83fa9970, mds: 0) 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.086405+0000 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.081820+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.081820+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b8404ede0 con 0x560b83ee4360 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400acc0 con 0x560b83ee4360 2024-03-28T01:08:24.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.085+0000 7f9dfca00640 10 client.4722 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de802bcb0 con 0x560b83ee4360 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.086405+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 features 0xffffffffffffffff 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 update_snap_trace len 70 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 1 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 15 client.4722 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 15 client.4722 snapc 1=[] 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 2 -> 1 2024-03-28T01:08:24.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 hrm is_target=1 is_dentry=0 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400afd0) - ctime 2024-03-28T01:08:24.067009+0000 mtime 2024-03-28T01:08:24.067009+0000 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 dir hash is 2 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 27 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 12 client.4722 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=- has_dir_layout 0x7f9dc400afd0) caps pAsLsXsFs 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 1 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 2 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 15 client.4722 add_update_cap first one, opened snaprealm 0x7f9dc400aee0 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 2 -> 1 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 lat 0.000510 2024-03-28T01:08:24.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 _ll_get 0x7f9dc400afd0 0x1 -> 1 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 3 client.4722 openat enter(AlternateNameRenameExistMatch_44790) 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 path_walk AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 0 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 (path is AlternateNameRenameExistMatch_44790) 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 may_lookup 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 _getattr mask As issued=1 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 3 client.4722 may_lookup 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 _do_lookup on #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_44790 => 2360090428 2024-03-28T01:08:24.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2360090428 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b8404ede0, mds: 0) 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.086992+0000 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:2 lookup #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.086966+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.086966+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84056300 con 0x560b83ee4360 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe68269330 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de802b400 con 0x560b83ee4360 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:24.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.086992+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 10 client.4722 insert_trace -- no trace 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe68269330 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 lat 0.000333 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 _do_lookup res is -2 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 _lookup 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_44790 = -2 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 path_walk cur=0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 path_walk 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 may_create 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 _getattr mask As issued=1 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 3 client.4722 may_create 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 8 client.4722 _create(0x1 AlternateNameRenameExistMatch_44790, 0777) 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 get_or_create 0x1.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 15 open_dir 0x560b83ee4820 on 0x7f9dc400afd0 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 15 client.4722 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMatch_44790' dn 0x1.head["AlternateNameRenameExistMatch_44790"] (new dn) 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_44790 => 2360090428 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2360090428 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b84056300, mds: 0) 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 encode_dentry_release enter(dn:0x7f9dc400abd0) 2024-03-28T01:08:24.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0), req:0x560b84056300 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.087451+0000 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMatch_44790 (foo) 2024-03-28T01:08:24.087414+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMatch_44790 (foo) 2024-03-28T01:08:24.087414+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fad220 con 0x560b83ee4360 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe68269690 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.086+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 859+0+0 (crc 0 0 0) 0x7f9de800f4a0 con 0x560b83ee4360 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.087451+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 features 0xffffffffffffffff 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 update_snap_trace len 70 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 1 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 2 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1000000000f.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) - ctime 2024-03-28T01:08:24.087414+0000 mtime 2024-03-28T01:08:24.087414+0000 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 12 client.4722 add_update_inode adding 0x1000000000f.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=- objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 2 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 3 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 15 client.4722 add_update_cap first one, opened snaprealm 0x7f9dc400aee0 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x1000000000f.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXsFs 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.067009+0000 ctime=2024-03-28T01:08:24.067009+0000 change_attr=27 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.087414+0000 mtime 2024-03-28T01:08:24.087414+0000 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 dir hash is 2 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 27 , mds inodestat change_attr: 28 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.198 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 10 client.4722 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 12 client.4722 insert_dentry_inode 'AlternateNameRenameExistMatch_44790' vino 0x1000000000f.head in dir 0x1.head dn 0x7f9dc400abd0 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 15 client.4722 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMatch_44790' to inode 0x7f9dc400b5b0 dn 0x1.head["AlternateNameRenameExistMatch_44790"] (old dn) 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 link inode 0x7f9dc400b5b0 parents now 0x1.head["AlternateNameRenameExistMatch_44790"] 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 15 client.4722 update_dentry_lease 0x1.head["AlternateNameRenameExistMatch_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:24.087451+0000 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 3 -> 2 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe68269690 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 client.4722 delegated_inos: [] 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 client.4722 make_request created ino 0x1000000000f 2024-03-28T01:08:24.199 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 make_request target is 0x1000000000f.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 lat 0.005458 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x1000000000f.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x1000000000f.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 client.4722 _create_fh 0x1000000000f mode 2 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 trim_cache size 1 max 16384 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 8 client.4722 create(#0x1/AlternateNameRenameExistMatch_44790, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 3 client.4722 openat exit(AlternateNameRenameExistMatch_44790) 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 client.4722 get_quota_root realm 0x1 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 client.4722 get_quota_root 0x1000000000f.head -> 0x1.head 2024-03-28T01:08:24.200 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 10 client.4722 cur file size is 0 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83f29150 0x560b83f295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- osd_op(unknown.0.0:1 3.12 3:49e35fbc:::1000000000f.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f1dd40 con 0x560b83f29150 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- osd_op(unknown.0.0:2 3.12 3:49e35fbc:::1000000000f.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f1e600 con 0x560b83f29150 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.092+0000 7f9e00600640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83f29150 0x560b83f295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.093+0000 7f9e00600640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83f29150 0x560b83f295d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=a9c6f47ee2a2650d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.093+0000 7f9dfca00640 10 client.4722 ms_handle_connect on v2:172.21.15.33:6817/1178911672 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.093+0000 7f9e00600640 1 -- 172.21.15.33:0/3250881336 <== osd.0 v2:172.21.15.33:6817/1178911672 1 ==== osd_op_reply(1 1000000000f.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83f29150 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e00600640 1 -- 172.21.15.33:0/3250881336 <== osd.0 v2:172.21.15.33:6817/1178911672 2 ==== osd_op_reply(2 1000000000f.00000000 [create] v23'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83f29150 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 wanted_max_size 0 -> 3 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 get_caps 0x1000000000f.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 clear_suid_sgid 0x1000000000f.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:24.201 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 5 client.4722 get_cap_ref got first FILE_BUFFER ref on 0x1000000000f.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 5 client.4722 get_cap_ref got first FILE_CACHE ref on 0x1000000000f.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000f.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) - -> Fw 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 cap_delay_requeue on 0x1000000000f.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 get_quota_root realm 0x1 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 get_quota_root 0x1000000000f.head -> 0x1.head 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 7 client.4722 wrote to 3, extending file size 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 mark_caps_dirty 0x1000000000f.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) Fw -> Fw 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 cap_delay_requeue on 0x1000000000f.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 3 client.4722 write(10, "...", 3, 0) = 3 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 3 client.4722 close enter(10) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 8 client.4722 _release_fh 0x560b8404f270 mode 2 on 0x1000000000f.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 _flush 0x1000000000f.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.202 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- osd_op(unknown.0.0:3 3.12 3:49e35fbc:::1000000000f.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f1dd40 con 0x560b83f29150 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 check_caps on 0x1000000000f.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 delaying cap release 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 cap_delay_requeue on 0x1000000000f.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 _release_fh 0x560b8404f270 on inode 0x1000000000f.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) no async_err state 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 3 client.4722 close exit(10) 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 3 client.4722 openat enter(AlternateNameRenameExistMatch_447902) 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 path_walk cur=0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 path_walk AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 0 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 (path is AlternateNameRenameExistMatch_447902) 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 may_lookup 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 _getattr mask As issued=1 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 3 client.4722 may_lookup 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 _do_lookup on #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_447902 => 477038804 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=477038804 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b83fad220, mds: 0) 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.097561+0000 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:4 lookup #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.097529+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 lookup #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.097529+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1dd40 con 0x560b83ee4360 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.096+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe68269330 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_reply(???:4 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de8000dd0 con 0x560b83ee4360 2024-03-28T01:08:24.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.097561+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9dfca00640 10 client.4722 insert_trace -- no trace 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe68269330 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 lat 0.000505 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 10 client.4722 _do_lookup res is -2 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 10 client.4722 _lookup 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_447902 = -2 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 path_walk cur=0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 10 client.4722 path_walk 2024-03-28T01:08:24.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 may_create 0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 10 client.4722 _getattr mask As issued=1 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 3 client.4722 may_create 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 8 client.4722 _create(0x1 AlternateNameRenameExistMatch_447902, 0777) 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 get_or_create 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 15 client.4722 link dir 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMatch_447902' dn 0x1.head["AlternateNameRenameExistMatch_447902"] (new dn) 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_447902 => 477038804 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=477038804 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 10 client.4722 choose_target_mds from dirfragtree hash 2024-03-28T01:08:24.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 5 for mds.0 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b83f1dd40, mds: 0) 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 encode_dentry_release enter(dn:0x560b83e5b530) 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0), req:0x560b83f1dd40 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.098153+0000 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:5 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMatch_447902 (foo2) 2024-03-28T01:08:24.098134+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:5 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMatch_447902 (foo2) 2024-03-28T01:08:24.098134+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1e150 con 0x560b83ee4360 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe68269690 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.097+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:5 = 0 (0) Success unsafe) v1 ==== 877+0+0 (crc 0 0 0) 0x7f9de8000fc0 con 0x560b83ee4360 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:0 tid 5 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.098153+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 features 0xffffffffffffffff 2024-03-28T01:08:24.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 update_snap_trace len 70 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 2 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 3 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x10000000010.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) - ctime 2024-03-28T01:08:24.098134+0000 mtime 2024-03-28T01:08:24.098134+0000 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 12 client.4722 add_update_inode adding 0x10000000010.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=- objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 3 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 4 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 15 client.4722 add_update_cap first one, opened snaprealm 0x7f9dc400aee0 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000010.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXs 2024-03-28T01:08:24.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.087414+0000 ctime=2024-03-28T01:08:24.087414+0000 change_attr=28 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.098134+0000 mtime 2024-03-28T01:08:24.098134+0000 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 dir hash is 2 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 28 , mds inodestat change_attr: 29 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 12 client.4722 insert_dentry_inode 'AlternateNameRenameExistMatch_447902' vino 0x10000000010.head in dir 0x1.head dn 0x560b83e5b530 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 15 client.4722 link dir 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMatch_447902' to inode 0x7f9dc400bb90 dn 0x1.head["AlternateNameRenameExistMatch_447902"] (old dn) 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 link inode 0x7f9dc400bb90 parents now 0x1.head["AlternateNameRenameExistMatch_447902"] 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 15 client.4722 update_dentry_lease 0x1.head["AlternateNameRenameExistMatch_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.098153+0000 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 4 -> 3 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe68269690 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 5 0x7f9dfc9fde80 2024-03-28T01:08:24.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 5 0x7f9dfc9fde80 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 delegated_inos: [0x10000000011~0x1f4] 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 make_request created ino 0x10000000010 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 make_request target is 0x10000000010.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 lat 0.001120 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000010.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000010.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 _create_fh 0x10000000010 mode 2 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 trim_cache size 2 max 16384 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 8 client.4722 create(#0x1/AlternateNameRenameExistMatch_447902, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 3 client.4722 openat exit(AlternateNameRenameExistMatch_447902) 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 get_quota_root realm 0x1 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 get_quota_root 0x10000000010.head -> 0x1.head 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 cur file size is 0 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 wanted_max_size 0 -> 3 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 get_caps 0x10000000010.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 clear_suid_sgid 0x10000000010.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 snaprealm snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 5 client.4722 get_cap_ref got first FILE_BUFFER ref on 0x10000000010.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 5 client.4722 get_cap_ref got first FILE_CACHE ref on 0x10000000010.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000010.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) - -> Fw 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 cap_delay_requeue on 0x10000000010.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 get_quota_root realm 0x1 2024-03-28T01:08:24.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 get_quota_root 0x10000000010.head -> 0x1.head 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 7 client.4722 wrote to 3, extending file size 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000010.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) Fw -> Fw 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 cap_delay_requeue on 0x10000000010.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 3 client.4722 write(10, "...", 3, 0) = 3 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 3 client.4722 close enter(10) 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 8 client.4722 _release_fh 0x560b8404f270 mode 2 on 0x10000000010.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 _flush 0x10000000010.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f1e570 0x560b83f223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:4 3.3b 3:dde6a3df:::10000000010.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f22930 con 0x560b83f1e570 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 check_caps on 0x10000000010.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 delaying cap release 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 cap_delay_requeue on 0x10000000010.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 _release_fh 0x560b8404f270 on inode 0x10000000010.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) no async_err state 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 3 client.4722 close exit(10) 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 path_walk cur=0x1.head(faked_ino=0 nref=26 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 path_walk 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 path_walk cur=0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 path_walk 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 may_delete 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0); ; name AlternateNameRenameExistMatch_44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 _getattr mask As issued=1 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 3 client.4722 may_delete 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 may_delete 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0); ; name AlternateNameRenameExistMatch_447902; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 _getattr mask As issued=1 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 3 client.4722 may_delete 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 8 client.4722 _rename(0x1 AlternateNameRenameExistMatch_44790 to 0x1 AlternateNameRenameExistMatch_447902 uid 0 gid 0) 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 get_or_create 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 get_or_create 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 _lookup have 0x1.head["AlternateNameRenameExistMatch_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 _do_lookup on #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_44790 => 2360090428 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2360090428 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 6 for mds.0 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b83f23140, mds: 0) 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.099569+0000 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:6 lookup #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.099551+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:6 lookup #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.099551+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f235d0 con 0x560b83ee4360 2024-03-28T01:08:24.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.098+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe682695a0 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_reply(???:6 = 0 (0) Success) v1 ==== 837+0+0 (crc 0 0 0) 0x7f9de800f830 con 0x560b83ee4360 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 6 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.099569+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 features 0xffffffffffffffff 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 update_snap_trace len 70 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 3 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 4 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1000000000f.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 0 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsxLsXsxFsxcrwb -> pAsxLsXsxFsxcrwb from mds.0 on 0x1000000000f.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXs 2024-03-28T01:08:24.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.098134+0000 mtime 2024-03-28T01:08:24.098134+0000 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 dir hash is 2 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 29 , mds inodestat change_attr: 29 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f1e570 0x560b83f223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 insert_dentry_inode 'AlternateNameRenameExistMatch_44790' vino 0x1000000000f.head in dir 0x1.head dn 0x7f9dc400abd0 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 had dentry AlternateNameRenameExistMatch_44790 with correct vino 0x1000000000f.head 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 15 client.4722 update_dentry_lease 0x1.head["AlternateNameRenameExistMatch_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:24.099569+0000 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 4 -> 3 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe682695a0 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 6 0x7f9dfc9fde80 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 6 0x7f9dfc9fde80 2024-03-28T01:08:24.216 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 make_request target is 0x1000000000f.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 lat 0.000521 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 _do_lookup res is 0 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 _lookup have 0x1.head["AlternateNameRenameExistMatch_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 _lookup 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_44790 = 0x1000000000f.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x1000000000f.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x1000000000f.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 _lookup have 0x1.head["AlternateNameRenameExistMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 _do_lookup on #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_447902 => 477038804 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=477038804 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.217 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 7 for mds.0 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b83f235d0, mds: 0) 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.100131+0000 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:7 lookup #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.100115+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:7 lookup #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.100115+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f23a50 con 0x560b83ee4360 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe682695a0 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:7 = 0 (0) Success) v1 ==== 839+0+0 (crc 0 0 0) 0x7f9de8038800 con 0x560b83ee4360 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 7 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.100131+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 features 0xffffffffffffffff 2024-03-28T01:08:24.218 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 update_snap_trace len 70 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 3 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 4 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x10000000010.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 0 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsxLsXsxFsxcrwb -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000010.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXs 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.098134+0000 mtime 2024-03-28T01:08:24.098134+0000 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 dir hash is 2 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 29 , mds inodestat change_attr: 29 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.219 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 insert_dentry_inode 'AlternateNameRenameExistMatch_447902' vino 0x10000000010.head in dir 0x1.head dn 0x560b83e5b530 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 12 client.4722 had dentry AlternateNameRenameExistMatch_447902 with correct vino 0x10000000010.head 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 15 client.4722 update_dentry_lease 0x1.head["AlternateNameRenameExistMatch_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.100131+0000 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 4 -> 3 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe682695a0 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 7 0x7f9dfc9fde80 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 7 0x7f9dfc9fde80 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 make_request target is 0x10000000010.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 lat 0.000481 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 _do_lookup res is 0 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 _lookup have 0x1.head["AlternateNameRenameExistMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 client.4722 _lookup 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_447902 = 0x10000000010.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.220 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.099+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_447902 => 477038804 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=477038804 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 10 client.4722 choose_target_mds from dirfragtree hash 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 8 for mds.0 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b83f1e150, mds: 0) 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 encode_inode_release enter(in:0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0), req:0x560b83f1e150 mds:0, drop:Ls, unless:-, force:0) 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 encode_inode_release enter(in:0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90), req:0x560b83f1e150 mds:0, drop:Lsx, unless:-, force:0) 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 encode_dentry_release enter(dn:0x560b83e5b530) 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0), req:0x560b83f1e150 mds:0, drop:XxFs, unless:Fx, force:1) 2024-03-28T01:08:24.221 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 encode_dentry_release enter(dn:0x7f9dc400abd0) 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0), req:0x560b83f1e150 mds:0, drop:LxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.100729+0000 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:8 rename #0x1/AlternateNameRenameExistMatch_447902 (foo2) #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.100677+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:8 rename #0x1/AlternateNameRenameExistMatch_447902 (foo2) #0x1/AlternateNameRenameExistMatch_44790 2024-03-28T01:08:24.100677+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f23a50 con 0x560b83ee4360 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe682697f0 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9e00600640 1 -- 172.21.15.33:0/3250881336 <== osd.0 v2:172.21.15.33:6817/1178911672 3 ==== osd_op_reply(3 1000000000f.00000000 [write 0~3] v23'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83f29150 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9df7400640 10 client.4722 _flushed 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9df7400640 5 client.4722 put_cap_ref dropped last FILE_BUFFER ref on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9df7400640 5 client.4722 put_cap_ref dropped last FILE_CACHE ref on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.222 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f1e570 0x560b83f223f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=b57ba2baba0605c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 ms_handle_connect on v2:172.21.15.194:6824/2849955959 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9df7400640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 2 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9df7400640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(revoke ino 0x1000000000f 1 seq 3 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.087414+0000 ctime 2024-03-28T01:08:24.087414+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de800a040 con 0x560b83ee4360 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 5 client.4722 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 mds.0 seq now 1 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 5 client.4722 handle_cap_grant on in 0x1000000000f mds.0 seq 3 caps now pAsXsFsxcrwb was pAsxXsxFsxcrwb 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) pAsxXsxFsxcrwb ctime 2024-03-28T01:08:24.087414+0000 mtime 2024-03-28T01:08:24.087414+0000 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 revocation of AxXx 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 check_caps on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) wanted - used Fc issued pAsXsFsxcrwb revoking AxXx flags=1 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsxFsxcrwb revoking AxXx 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 completed revocation of AxXx 2024-03-28T01:08:24.223 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 mark_caps_flushing Fw 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 mark_caps_clean 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 send_cap 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) mds.0 seq 3 used Fc want - flush Fw retain pAsLsXsFsc held pAsxXsxFsxcrwb revoking AxXx dropping Fxrwb 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 15 client.4722 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x1000000000f 1 seq 3 tid 2 caps=pAsXsFsc dirty=Fw wanted=- follows 1 size 3/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.097344+0000 ctime 2024-03-28T01:08:24.097344+0000 change_attr 1) v12 -- 0x7f9dc400c170 con 0x560b83ee4360 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_caps(revoke ino 0x10000000010 1 seq 3 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.098134+0000 ctime 2024-03-28T01:08:24.098134+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de804e040 con 0x560b83ee4360 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 mds.0 seq now 2 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 5 client.4722 handle_cap_grant on in 0x10000000010 mds.0 seq 3 caps now pAsXsFsxcrwb was pAsxXsxFsxcrwb 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) pAsxXsxFsxcrwb ctime 2024-03-28T01:08:24.098134+0000 mtime 2024-03-28T01:08:24.098134+0000 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 revocation of AxXx 2024-03-28T01:08:24.224 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 check_caps on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) wanted Fxcb used Fcb issued pAsXsFsxcrwb revoking AxXx flags=1 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsxFsxcrwb revoking AxXx 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 completed revocation of AxXx 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 mark_caps_flushing Fw 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 mark_caps_clean 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 send_cap 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) mds.0 seq 3 used Fcb want Fxcb flush Fw retain pAsLsxXsFsxcrwbl held pAsxXsxFsxcrwb revoking AxXx dropping - 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 15 client.4722 auth cap, requesting max_size 3 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000010 1 seq 3 tid 3 caps=pAsXsFsxcrwb dirty=Fw wanted=Fxcb follows 1 size 3/3 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.099386+0000 ctime 2024-03-28T01:08:24.099386+0000 change_attr 1) v12 -- 0x7f9de800a040 con 0x560b83ee4360 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.100+0000 7f9dfca00640 10 client.4722 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.103+0000 7f9dffc00640 1 -- 172.21.15.33:0/3250881336 <== osd.7 v2:172.21.15.194:6824/2849955959 1 ==== osd_op_reply(4 10000000010.00000000 [write 0~3] v23'1 uv1 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df00b6020 con 0x560b83f1e570 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.103+0000 7f9df7400640 10 client.4722 _flushed 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.103+0000 7f9df7400640 5 client.4722 put_cap_ref dropped last FILE_BUFFER ref on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.103+0000 7f9df7400640 5 client.4722 put_cap_ref dropped last FILE_CACHE ref on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.103+0000 7f9df7400640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 2 2024-03-28T01:08:24.225 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.103+0000 7f9df7400640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.104+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 12 ==== client_caps(grant ino 0x10000000010 1 seq 4 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 3/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.099386+0000 ctime 2024-03-28T01:08:24.099386+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de800a040 con 0x560b83ee4360 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.104+0000 7f9dfca00640 10 client.4722 mds.0 seq now 3 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.104+0000 7f9dfca00640 5 client.4722 handle_cap_grant on in 0x10000000010 mds.0 seq 4 caps now pAsXsFsxcrwb was pAsXsFsxcrwb 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.104+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x10000000010.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400bb90) pAsXsFsxcrwb ctime 2024-03-28T01:08:24.099386+0000 mtime 2024-03-28T01:08:24.099386+0000 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.104+0000 7f9dfca00640 10 client.4722 caps unchanged at pAsXsFsxcrwb 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.104+0000 7f9dfca00640 10 client.4722 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 13 ==== client_reply(???:8 = 0 (0) Success unsafe) v1 ==== 839+0+0 (crc 0 0 0) 0x7f9de8042800 con 0x560b83ee4360 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:0 tid 8 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.100729+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 features 0xffffffffffffffff 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 update_snap_trace len 70 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 3 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 4 2024-03-28T01:08:24.226 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsXsFsc(0=pAsXsFsc) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) caps pAsLsXsFsxcrwb 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.097344+0000 change_attr=1 caps=pAsXsFsc(0=pAsXsFsc) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) pAsXsFscw ctime 2024-03-28T01:08:24.100677+0000 mtime 2024-03-28T01:08:24.097344+0000 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 2 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsXsFsc -> pAsLsXsFsxcrwb from mds.0 on 0x1000000000f.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXs 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.098134+0000 ctime=2024-03-28T01:08:24.098134+0000 change_attr=29 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.100677+0000 mtime 2024-03-28T01:08:24.100677+0000 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 20 client.4722 dir hash is 2 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 29 , mds inodestat change_attr: 32 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.106+0000 7f9dfca00640 20 client.4722 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 12 client.4722 insert_dentry_inode 'AlternateNameRenameExistMatch_447902' vino 0x1000000000f.head in dir 0x1.head dn 0x560b83e5b530 2024-03-28T01:08:24.227 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 12 client.4722 had dentry AlternateNameRenameExistMatch_447902 with WRONG vino 0x10000000010.head 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 15 client.4722 unlink dir 0x7f9dc400afd0 'AlternateNameRenameExistMatch_447902' dn 0x560b83e5b530 inode 0x7f9dc400bb90 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 unlink inode 0x7f9dc400bb90 parents now 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 15 client.4722 unlink dir 0x7f9dc400afd0 'AlternateNameRenameExistMatch_44790' dn 0x7f9dc400abd0 inode 0x7f9dc400b5b0 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 unlink inode 0x7f9dc400b5b0 parents now 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 15 client.4722 unlink removing 'AlternateNameRenameExistMatch_44790' dn 0x7f9dc400abd0 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 15 client.4722 link dir 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMatch_447902' to inode 0x7f9dc400b5b0 dn 0x1.head["AlternateNameRenameExistMatch_447902"] (old dn) 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 link inode 0x7f9dc400b5b0 parents now 0x1.head["AlternateNameRenameExistMatch_447902"] 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.228 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 15 client.4722 update_dentry_lease 0x1.head["AlternateNameRenameExistMatch_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.100729+0000 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 4 -> 3 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe682697f0 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 8 0x7f9dfc9fde80 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 8 0x7f9dfc9fde80 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 make_request target is 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 lat 0.007029 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 rename result is 0 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 trim_cache size 1 max 16384 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 8 client.4722 _rename(#0x1/AlternateNameRenameExistMatch_44790, #0x1/AlternateNameRenameExistMatch_447902) = 0 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 walk: AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.229 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 path_walk cur=0x1.head(faked_ino=0 nref=31 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 path_walk AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 14 ==== client_reply(???:5 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de8042a10 con 0x560b83ee4360 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 0 0x1.head(faked_ino=0 nref=31 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 (path is AlternateNameRenameExistMatch_447902) 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 may_lookup 0x1.head(faked_ino=0 nref=31 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 _getattr mask As issued=1 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 3 client.4722 may_lookup 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 _lookup have 0x1.head["AlternateNameRenameExistMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 _do_lookup on #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMatch_447902 => 477038804 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 choose_target_mds 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=477038804 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.230 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 mds is 0 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 send_request rebuilding request 9 for mds.0 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 encode_cap_releases enter (req: 0x560b83f23a50, mds: 0) 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 send_request set sent_stamp to 2024-03-28T01:08:24.107811+0000 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 10 client.4722 send_request client_request(unknown.0:9 lookup #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.107797+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:9 lookup #0x1/AlternateNameRenameExistMatch_447902 2024-03-28T01:08:24.107797+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1fa20 con 0x560b83ee4360 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9e0324a480 20 client.4722 awaiting reply|forward|kick on 0x7ffe682695f0 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 5 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.098153+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 10 client.4722 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.107+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 15 ==== client_caps(flush_ack ino 0x1000000000f 1 seq 3 tid 2 caps=pAsXsFsc dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400c170 con 0x560b83ee4360 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 mds.0 seq now 4 2024-03-28T01:08:24.231 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 5 client.4722 handle_cap_flush_ack mds.0 cleaned Fw on 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) with Fw 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 5 client.4722 flushing_caps Fw -> - 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) !flushing 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 16 ==== client_caps(grant ino 0x1000000000f 1 seq 5 caps=pAsLsXsFsxcrwb dirty=- wanted=- follows 0 size 3/0 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.097344+0000 ctime 2024-03-28T01:08:24.100677+0000 change_attr 2) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de8042a10 con 0x560b83ee4360 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 mds.0 seq now 5 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 5 client.4722 handle_cap_grant on in 0x1000000000f mds.0 seq 5 caps now pAsLsXsFsxcrwb was pAsLsXsFsxcrwb 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) pAsLsXsFsxcrwb ctime 2024-03-28T01:08:24.100677+0000 mtime 2024-03-28T01:08:24.097344+0000 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 caps unchanged at pAsLsXsFsxcrwb 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 17 ==== client_caps(flush_ack ino 0x10000000010 1 seq 3 tid 3 caps=pAsXsFsxcrwb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de8077450 con 0x560b83ee4360 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 mds.0 seq now 6 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 5 client.4722 handle_cap_flush_ack mds.0 cleaned Fw on 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) with Fw 2024-03-28T01:08:24.232 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 5 client.4722 flushing_caps Fw -> - 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) !flushing 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 10 client.4722 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.111+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 18 ==== client_reply(???:9 = 0 (0) Success) v1 ==== 839+0+0 (crc 0 0 0) 0x7f9de8077b60 con 0x560b83ee4360 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 9 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.107811+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 features 0xffffffffffffffff 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 update_snap_trace len 70 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref was 3 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 get_snap_realm 0x1 0x7f9dc400aee0, nref now is 4 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) caps pAsLsXsFsxcrwb 2024-03-28T01:08:24.233 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) pAsLsXsFsxcrwb ctime 2024-03-28T01:08:24.100677+0000 mtime 2024-03-28T01:08:24.097344+0000 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 2 , mds inodestat change_attr: 2 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsLsXsFsxcrwb -> pAsLsXsFsxcrwb from mds.0 on 0x1000000000f.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 12 client.4722 add_update_inode had 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXsFs 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.100677+0000 mtime 2024-03-28T01:08:24.100677+0000 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 dir hash is 2 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 12 client.4722 add_update_inode client inode change_attr: 32 , mds inodestat change_attr: 32 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 12 client.4722 insert_dentry_inode 'AlternateNameRenameExistMatch_447902' vino 0x1000000000f.head in dir 0x1.head dn 0x560b83e5b530 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 12 client.4722 had dentry AlternateNameRenameExistMatch_447902 with correct vino 0x1000000000f.head 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 15 client.4722 update_dentry_lease 0x1.head["AlternateNameRenameExistMatch_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:24.107811+0000 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 4 -> 3 2024-03-28T01:08:24.234 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 handle_client_reply signalling caller 0x7ffe682695f0 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 handle_client_reply awaiting kickback on tid 9 0x7f9dfc9fde80 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 20 client.4722 sendrecv kickback on tid 9 0x7f9dfc9fde80 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 20 client.4722 make_request target is 0x1000000000f.head(faked_ino=0 nref=25 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 20 client.4722 lat 0.005582 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 10 client.4722 _do_lookup res is 0 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 20 client.4722 _lookup have 0x1.head["AlternateNameRenameExistMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 10 client.4722 _lookup 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMatch_447902 = 0x1000000000f.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 2 client.4722 unmount 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.235 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, size was 1+3 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 trim_cache size 1 max 16384 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, size still 1+3 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 2 client.4722 unmounting 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b8404f860 con 0x560b83ee4360 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9e0324a480 10 client.4722 waiting for write request '8' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 19 ==== client_caps(grant ino 0x10000000010 1 seq 5 caps=pAsLsXsFsxcrwb dirty=- wanted=LsFxcb follows 0 size 3/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.099386+0000 ctime 2024-03-28T01:08:24.100677+0000 change_attr 2) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de8060440 con 0x560b83ee4360 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 mds.0 seq now 7 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 5 client.4722 handle_cap_grant on in 0x10000000010 mds.0 seq 5 caps now pAsLsXsFsxcrwb was pAsXsFsxcrwb 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 update_inode_file_time 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.099386+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) pAsXsFsxcrwb ctime 2024-03-28T01:08:24.100677+0000 mtime 2024-03-28T01:08:24.099386+0000 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 grant, new caps are Ls 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 5 client.4722 _try_to_trim_inode in 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.236 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, size was 1+3 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 trim_cache size 1 max 16384 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, size still 1+3 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 20 ==== client_reply(???:8 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de80607a0 con 0x560b83ee4360 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 handle_client_reply got a reply. Safe:1 tid 8 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 insert_trace from 2024-03-28T01:08:24.100729+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 10 client.4722 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.112+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=26 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMatch_447902"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 5 client.4722 _try_to_trim_inode in 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, size was 1+3 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 trim_cache size 1 max 16384 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 15 client.4722 trim_dentry unlinking dn AlternateNameRenameExistMatch_447902 in dir 0x1 2024-03-28T01:08:24.237 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 15 client.4722 unlink dir 0x7f9dc400afd0 'AlternateNameRenameExistMatch_447902' dn 0x560b83e5b530 inode 0x7f9dc400b5b0 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 unlink inode 0x7f9dc400b5b0 parents now 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 15 client.4722 unlink removing 'AlternateNameRenameExistMatch_447902' dn 0x560b83e5b530 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 15 client.4722 close_dir dir 0x560b83ee4820 on 0x7f9dc400afd0 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=27 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _ll_drop_pins 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 _ll_put 0x7f9dc400afd0 0x1 1 -> 0 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=33 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=33 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.238 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 _release 0x10000000010.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _invalidate_inode_cache 0x10000000010.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _flush 0x10000000010.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 nothing to flush 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 _release 0x1000000000f.head(faked_ino=0 nref=28 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _invalidate_inode_cache 0x1000000000f.head(faked_ino=0 nref=28 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _flush 0x1000000000f.head(faked_ino=0 nref=29 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 nothing to flush 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=29 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 _release 0x1.head(faked_ino=0 nref=34 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=34 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _flush 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.239 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 nothing to flush 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x10000000010.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1000000000f.head(faked_ino=0 nref=29 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_inode on 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 flush_caps_sync 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 wait_sync_caps want 3 (last is 3, 0 total flushing) 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 trim_cache size 0 max 16384 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _put_inode on 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 34 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 15 client.4722 remove_cap last one, closing snaprealm 0x7f9dc400aee0 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 3 -> 2 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=- has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _put_inode on 0x1000000000f.head(faked_ino=0 nref=29 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 28 2024-03-28T01:08:24.240 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 remove_cap mds.0 on 0x1000000000f.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 15 client.4722 remove_cap last one, closing snaprealm 0x7f9dc400aee0 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 2 -> 1 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _put_inode deleting 0x1000000000f.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/0 nlink=1 btime=2024-03-28T01:08:24.087414+0000 mtime=2024-03-28T01:08:24.097344+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=- objectset[0x1000000000f ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _put_inode on 0x10000000010.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 18 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 remove_cap mds.0 on 0x10000000010.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 15 client.4722 remove_cap last one, closing snaprealm 0x7f9dc400aee0 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 20 client.4722 put_snap_realm 0x1 0x7f9dc400aee0 1 -> 0 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 10 client.4722 _put_inode deleting 0x10000000010.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=0 btime=2024-03-28T01:08:24.098134+0000 mtime=2024-03-28T01:08:24.099386+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=2 caps=- objectset[0x10000000010 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 2 client.4722 _close_mds_session mds.0 seq 7 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 7) v7 -- 0x560b83faa800 con 0x560b83ee4360 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.113+0000 7f9e0324a480 2 client.4722 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 <== mds.0 v2:172.21.15.33:6834/2290842450 21 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b8404f860 con 0x560b83ee4360 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 10 client.4722 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:24.241 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 5 client.4722 _closed_mds_session mds.0 seq 7 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 1 -- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83ee4360 msgr2=0x560b83fb4070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b83ee4360 0x560b83fb4070 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 10 client.4722 remove_session_caps mds.0 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 10 client.4722 kick_requests_closed for mds.0 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, size was 0+0 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 20 client.4722 trim_cache size 0 max 16384 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9dfca00640 10 client.4722 unmounting: trim pass, size still 0+0 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9e0324a480 2 client.4722 unmounted. 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.147+0000 7f9e0324a480 1 client.4722 shutdown 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83f29150 msgr2=0x560b83f295d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x560b83f29150 0x560b83f295d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f1e570 msgr2=0x560b83f223f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83f1e570 0x560b83f223f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.242 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8406ccb0 msgr2=0x560b83fba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3250881336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b8406ccb0 0x560b83fba320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9df0017f80 tx=0x7f9df00ac000 comp rx=0 tx=0).stop 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e00600640 1 -- 172.21.15.33:0/3250881336 reap_dead start 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 shutdown_connections 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.153+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 >> 172.21.15.33:0/3250881336 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.154+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 shutdown_connections 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.154+0000 7f9e0324a480 1 -- 172.21.15.33:0/3250881336 wait complete. 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.154+0000 7f9e0324a480 20 client.4722 trim_cache size 0 max 16384 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.AlternateNameRenameExistMatch (78 ms) 2024-03-28T01:08:24.243 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameRenameExistMisMatch 2024-03-28T01:08:24.257 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.156+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:24.257 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.157+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.159+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9970 0x560b83fba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.160+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.160+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f99a60 0x560b83f2ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.160+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4005c80 con 0x560b83fa9970 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.160+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83fbee40 con 0x560b83f99a60 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.160+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b83f2ddd0 con 0x560b83fad220 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.160+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9970 0x560b83fba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.161+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9970 0x560b83fba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60992/0 (socket says 172.21.15.33:60992) 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.161+0000 7f9dffc00640 1 -- 172.21.15.33:0/2612231146 learned_addr learned my addr 172.21.15.33:0/2612231146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.161+0000 7f9dffc00640 1 -- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.161+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b84059ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.161+0000 7f9dffc00640 1 -- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f99a60 msgr2=0x560b83f2ca90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.161+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83f99a60 0x560b83f2ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:24.258 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.161+0000 7f9dffc00640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83fa9e90 con 0x560b83fa9970 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.162+0000 7f9dffc00640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9970 0x560b83fba320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9df001bd70 tx=0x560b83fbab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096b6974b01557f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.162+0000 7f9dfde00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5070 con 0x560b83fa9970 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9df004ae40 con 0x560b83fa9970 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9e0324a480 10 client.4725 Subscribing to map 'mdsmap' 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b83f1d6b0 con 0x560b83fa9970 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9df5600640 20 client.4725 tick 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9df5600640 20 client.4725 collect_and_send_metrics 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9df5600640 20 client.4725 collect_and_send_global_metrics 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9df5600640 5 client.4725 collect_and_send_global_metrics MDS rank 0 is not ready yet -- not sending metric 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9df5600640 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.163+0000 7f9df5600640 20 client.4725 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.168+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b83fae340 con 0x560b83fa9970 2024-03-28T01:08:24.259 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.168+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9df00a1070 con 0x560b83fa9970 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9dfde00640 1 client.4725 handle_mds_map epoch 8 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9e0324a480 20 client.4725 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9e0324a480 10 client.4725 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9e0324a480 10 client.4725 _open_mds_session mds.0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406bdf0 0x560b83ee4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faa800 con 0x560b8406bdf0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.169+0000 7f9e0324a480 10 client.4725 waiting for session to mds.0 to open 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.170+0000 7f9dff200640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406bdf0 0x560b83ee4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.171+0000 7f9dff200640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406bdf0 0x560b83ee4360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b8ffea2a71083f server_cookie=ba944250cccd6afd in_seq=0 out_seq=0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.171+0000 7f9dfde00640 10 client.4725 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faa800 con 0x560b8406bdf0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9dfde00640 10 client.4725 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:24.260 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9dfde00640 10 client.4725 renew_caps mds.0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc4008d30 con 0x560b8406bdf0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9dfde00640 10 client.4725 connect_mds_targets for mds.0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 10 client.4725 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b83fa5ce0, mds: 0) 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.185339+0000 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.169770+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:24.169770+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb47d0 con 0x560b8406bdf0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc4008d30 con 0x560b8406bdf0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.184+0000 7f9dfde00640 10 client.4725 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:24.261 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9de800f4a0 con 0x560b8406bdf0 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.185339+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 features 0xffffffffffffffff 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 update_snap_trace len 70 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 1 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 15 client.4725 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 15 client.4725 snapc 1=[] 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 2 -> 1 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 hrm is_target=1 is_dentry=0 2024-03-28T01:08:24.262 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400afd0) - ctime 2024-03-28T01:08:24.100677+0000 mtime 2024-03-28T01:08:24.100677+0000 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 dir hash is 2 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 32 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 12 client.4725 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=- has_dir_layout 0x7f9dc400afd0) caps pAsLsXsFs 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 1 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 2 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 15 client.4725 add_update_cap first one, opened snaprealm 0x7f9dc4008f50 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 2 -> 1 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 1 0x7f9dfddfde80 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 lat 0.000745 2024-03-28T01:08:24.263 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 _ll_get 0x7f9dc400afd0 0x1 -> 1 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 3 client.4725 openat enter(AlternateNameRenameExistMisMatch_44790) 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 path_walk AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 0 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 (path is AlternateNameRenameExistMisMatch_44790) 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 may_lookup 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 _getattr mask As issued=1 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 3 client.4725 may_lookup 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 _do_lookup on #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.264 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_44790 => 2774950714 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2774950714 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b83fb47d0, mds: 0) 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.186149+0000 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:2 lookup #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.186134+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.186134+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f1dd40 con 0x560b8406bdf0 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe68269330 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de8000fe0 con 0x560b8406bdf0 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.186149+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 10 client.4725 insert_trace -- no trace 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe68269330 2024-03-28T01:08:24.265 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 2 0x7f9dfddfde80 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 lat 0.000373 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 _do_lookup res is -2 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 _lookup 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_44790 = -2 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 path_walk cur=0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 path_walk 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 may_create 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 _getattr mask As issued=1 2024-03-28T01:08:24.266 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 3 client.4725 may_create 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 8 client.4725 _create(0x1 AlternateNameRenameExistMisMatch_44790, 0777) 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 get_or_create 0x1.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 15 open_dir 0x560b83fbef20 on 0x7f9dc400afd0 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 15 client.4725 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMisMatch_44790' dn 0x1.head["AlternateNameRenameExistMisMatch_44790"] (new dn) 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_44790 => 2774950714 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2774950714 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b83f1dd40, mds: 0) 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 encode_dentry_release enter(dn:0x7f9dc4003190) 2024-03-28T01:08:24.267 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0), req:0x560b83f1dd40 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.186595+0000 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMisMatch_44790 (foo) 2024-03-28T01:08:24.186574+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMisMatch_44790 (foo) 2024-03-28T01:08:24.186574+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f23a50 con 0x560b8406bdf0 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe68269690 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.185+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 862+0+0 (crc 0 0 0) 0x7f9de802b8b0 con 0x560b8406bdf0 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.186595+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 features 0xffffffffffffffff 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 update_snap_trace len 70 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 1 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 2 2024-03-28T01:08:24.268 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x10000000011.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) - ctime 2024-03-28T01:08:24.186574+0000 mtime 2024-03-28T01:08:24.186574+0000 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 12 client.4725 add_update_inode adding 0x10000000011.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=- objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 2 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 3 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 15 client.4725 add_update_cap first one, opened snaprealm 0x7f9dc4008f50 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000011.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXsFs 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.100677+0000 ctime=2024-03-28T01:08:24.100677+0000 change_attr=32 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.186574+0000 mtime 2024-03-28T01:08:24.186574+0000 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 dir hash is 2 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 32 , mds inodestat change_attr: 33 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.269 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 10 client.4725 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 12 client.4725 insert_dentry_inode 'AlternateNameRenameExistMisMatch_44790' vino 0x10000000011.head in dir 0x1.head dn 0x7f9dc4003190 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 15 client.4725 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMisMatch_44790' to inode 0x7f9dc400b5b0 dn 0x1.head["AlternateNameRenameExistMisMatch_44790"] (old dn) 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 link inode 0x7f9dc400b5b0 parents now 0x1.head["AlternateNameRenameExistMisMatch_44790"] 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 15 client.4725 update_dentry_lease 0x1.head["AlternateNameRenameExistMisMatch_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:24.186595+0000 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 3 -> 2 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe68269690 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 3 0x7f9dfddfde80 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 client.4725 delegated_inos: [] 2024-03-28T01:08:24.270 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 client.4725 make_request created ino 0x10000000011 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 make_request target is 0x10000000011.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 lat 0.004401 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000011.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000011.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 client.4725 _create_fh 0x10000000011 mode 2 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 trim_cache size 1 max 16384 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 8 client.4725 create(#0x1/AlternateNameRenameExistMisMatch_44790, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 3 client.4725 openat exit(AlternateNameRenameExistMisMatch_44790) 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.271 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 client.4725 get_quota_root realm 0x1 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 client.4725 get_quota_root 0x10000000011.head -> 0x1.head 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 10 client.4725 cur file size is 0 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83eff1f0 0x560b84053c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:1 3.15 3:ab5910e2:::10000000011.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b8404ede0 con 0x560b83eff1f0 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.190+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:2 3.15 3:ab5910e2:::10000000011.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f23410 con 0x560b83eff1f0 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.191+0000 7f9e00600640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83eff1f0 0x560b84053c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.191+0000 7f9e00600640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83eff1f0 0x560b84053c50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=7fdb1e1f15c29f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.191+0000 7f9dfde00640 10 client.4725 ms_handle_connect on v2:172.21.15.194:6824/2849955959 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.191+0000 7f9e00600640 1 -- 172.21.15.33:0/2612231146 <== osd.7 v2:172.21.15.194:6824/2849955959 1 ==== osd_op_reply(1 10000000011.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83eff1f0 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.192+0000 7f9e00600640 1 -- 172.21.15.33:0/2612231146 <== osd.7 v2:172.21.15.194:6824/2849955959 2 ==== osd_op_reply(2 10000000011.00000000 [create] v23'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83eff1f0 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.192+0000 7f9e0324a480 10 client.4725 wanted_max_size 0 -> 3 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.192+0000 7f9e0324a480 10 client.4725 get_caps 0x10000000011.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:24.272 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.192+0000 7f9e0324a480 20 client.4725 clear_suid_sgid 0x10000000011.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.192+0000 7f9e0324a480 10 client.4725 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.192+0000 7f9e0324a480 5 client.4725 get_cap_ref got first FILE_BUFFER ref on 0x10000000011.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.192+0000 7f9e0324a480 5 client.4725 get_cap_ref got first FILE_CACHE ref on 0x10000000011.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000011.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) - -> Fw 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 cap_delay_requeue on 0x10000000011.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 get_quota_root realm 0x1 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 get_quota_root 0x10000000011.head -> 0x1.head 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 7 client.4725 wrote to 3, extending file size 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000011.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) Fw -> Fw 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 cap_delay_requeue on 0x10000000011.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 3 client.4725 write(10, "...", 3, 0) = 3 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 3 client.4725 close enter(10) 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 8 client.4725 _release_fh 0x560b83f23a50 mode 2 on 0x10000000011.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.273 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 _flush 0x10000000011.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:3 3.15 3:ab5910e2:::10000000011.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b8404ede0 con 0x560b83eff1f0 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 check_caps on 0x10000000011.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 delaying cap release 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 cap_delay_requeue on 0x10000000011.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 _release_fh 0x560b83f23a50 on inode 0x10000000011.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) no async_err state 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 3 client.4725 close exit(10) 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 3 client.4725 openat enter(AlternateNameRenameExistMisMatch_447902) 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 path_walk cur=0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 path_walk AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 0 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.274 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 (path is AlternateNameRenameExistMisMatch_447902) 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 may_lookup 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 _getattr mask As issued=1 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 3 client.4725 may_lookup 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 _do_lookup on #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_447902 => 2277440192 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2277440192 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b8404ede0, mds: 0) 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.193803+0000 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:4 lookup #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.193790+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.275 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 lookup #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.193790+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b84056930 con 0x560b8406bdf0 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe68269330 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_reply(???:4 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de802baa0 con 0x560b8406bdf0 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.193803+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9dfde00640 10 client.4725 insert_trace -- no trace 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe68269330 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 4 0x7f9dfddfde80 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 4 0x7f9dfddfde80 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 lat 0.000354 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 _do_lookup res is -2 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 _lookup 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_447902 = -2 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.276 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 path_walk cur=0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 path_walk 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 may_create 0x1.head(faked_ino=0 nref=24 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 _getattr mask As issued=1 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 3 client.4725 may_create 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 8 client.4725 _create(0x1 AlternateNameRenameExistMisMatch_447902, 0777) 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 get_or_create 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 15 client.4725 link dir 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMisMatch_447902' dn 0x1.head["AlternateNameRenameExistMisMatch_447902"] (new dn) 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_447902 => 2277440192 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2277440192 2024-03-28T01:08:24.277 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 choose_target_mds from dirfragtree hash 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 5 for mds.0 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b84056930, mds: 0) 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 encode_dentry_release enter(dn:0x560b83e53e70) 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0), req:0x560b84056930 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.194262+0000 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:5 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMisMatch_447902 (foomismatch) 2024-03-28T01:08:24.194233+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:5 create owner_uid=0, owner_gid=0 #0x1/AlternateNameRenameExistMisMatch_447902 (foomismatch) 2024-03-28T01:08:24.194233+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f26f10 con 0x560b8406bdf0 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe68269690 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.193+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:5 = 0 (0) Success unsafe) v1 ==== 887+0+0 (crc 0 0 0) 0x7f9de802bc90 con 0x560b8406bdf0 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:0 tid 5 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.194262+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.278 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 features 0xffffffffffffffff 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 update_snap_trace len 70 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 2 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 3 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x10000000012.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) - ctime 2024-03-28T01:08:24.194233+0000 mtime 2024-03-28T01:08:24.194233+0000 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode adding 0x10000000012.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=- objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 3 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 4 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 15 client.4725 add_update_cap first one, opened snaprealm 0x7f9dc4008f50 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000012.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXs 2024-03-28T01:08:24.279 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.186574+0000 ctime=2024-03-28T01:08:24.186574+0000 change_attr=33 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.194233+0000 mtime 2024-03-28T01:08:24.194233+0000 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 dir hash is 2 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 33 , mds inodestat change_attr: 34 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 insert_dentry_inode 'AlternateNameRenameExistMisMatch_447902' vino 0x10000000012.head in dir 0x1.head dn 0x560b83e53e70 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 15 client.4725 link dir 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 'AlternateNameRenameExistMisMatch_447902' to inode 0x7f9dc400bb90 dn 0x1.head["AlternateNameRenameExistMisMatch_447902"] (old dn) 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 link inode 0x7f9dc400bb90 parents now 0x1.head["AlternateNameRenameExistMisMatch_447902"] 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 15 client.4725 update_dentry_lease 0x1.head["AlternateNameRenameExistMisMatch_447902"] lease(mask 0 dur 0 altn foomismatch)) from 2024-03-28T01:08:24.194262+0000 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 4 -> 3 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe68269690 2024-03-28T01:08:24.280 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 5 0x7f9dfddfde80 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 5 0x7f9dfddfde80 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 delegated_inos: [0x10000000013~0x1f4] 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 make_request created ino 0x10000000012 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 make_request target is 0x10000000012.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 lat 0.001514 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000012.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000012.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 _create_fh 0x10000000012 mode 2 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 trim_cache size 2 max 16384 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 8 client.4725 create(#0x1/AlternateNameRenameExistMisMatch_447902, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=25 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.281 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 3 client.4725 openat exit(AlternateNameRenameExistMisMatch_447902) 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 get_quota_root realm 0x1 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 get_quota_root 0x10000000012.head -> 0x1.head 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 cur file size is 0 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 wanted_max_size 0 -> 3 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 get_caps 0x10000000012.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 clear_suid_sgid 0x10000000012.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 snaprealm snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 5 client.4725 get_cap_ref got first FILE_BUFFER ref on 0x10000000012.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 5 client.4725 get_cap_ref got first FILE_CACHE ref on 0x10000000012.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000012.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) - -> Fw 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 cap_delay_requeue on 0x10000000012.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 get_quota_root realm 0x1 2024-03-28T01:08:24.282 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 get_quota_root 0x10000000012.head -> 0x1.head 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 7 client.4725 wrote to 3, extending file size 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000012.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) Fw -> Fw 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 cap_delay_requeue on 0x10000000012.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 3 client.4725 write(10, "...", 3, 0) = 3 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 3 client.4725 close enter(10) 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 8 client.4725 _release_fh 0x560b83f23a50 mode 2 on 0x10000000012.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 _flush 0x10000000012.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- osd_op(unknown.0.0:4 3.d 3:b185e6e8:::10000000012.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83f27330 con 0x560b83eff1f0 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 check_caps on 0x10000000012.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 delaying cap release 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 cap_delay_requeue on 0x10000000012.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 _release_fh 0x560b83f23a50 on inode 0x10000000012.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) no async_err state 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.283 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 3 client.4725 close exit(10) 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 path_walk cur=0x1.head(faked_ino=0 nref=26 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 path_walk 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 path_walk cur=0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 path_walk 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 may_delete 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0); ; name AlternateNameRenameExistMisMatch_44790; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 _getattr mask As issued=1 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 3 client.4725 may_delete 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 may_delete 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0); ; name AlternateNameRenameExistMisMatch_447902; UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 _getattr mask As issued=1 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 3 client.4725 may_delete 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 8 client.4725 _rename(0x1 AlternateNameRenameExistMisMatch_44790 to 0x1 AlternateNameRenameExistMisMatch_447902 uid 0 gid 0) 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 get_or_create 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 get_or_create 0x1.head(faked_ino=0 nref=27 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) name AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.284 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 _lookup have 0x1.head["AlternateNameRenameExistMisMatch_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 _do_lookup on #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_44790 => 2774950714 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2774950714 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 6 for mds.0 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b83f220d0, mds: 0) 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.196006+0000 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:6 lookup #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.195991+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:6 lookup #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.195991+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f22560 con 0x560b8406bdf0 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe682695a0 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_reply(???:6 = 0 (0) Success) v1 ==== 840+0+0 (crc 0 0 0) 0x7f9de800f830 con 0x560b8406bdf0 2024-03-28T01:08:24.285 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 6 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.196006+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 features 0xffffffffffffffff 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 update_snap_trace len 70 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 3 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 4 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x10000000011.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 0 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsxLsXsxFsxcrwb -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000011.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXs 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.194233+0000 mtime 2024-03-28T01:08:24.194233+0000 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 dir hash is 2 2024-03-28T01:08:24.286 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 34 , mds inodestat change_attr: 34 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 insert_dentry_inode 'AlternateNameRenameExistMisMatch_44790' vino 0x10000000011.head in dir 0x1.head dn 0x7f9dc4003190 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 12 client.4725 had dentry AlternateNameRenameExistMisMatch_44790 with correct vino 0x10000000011.head 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 15 client.4725 update_dentry_lease 0x1.head["AlternateNameRenameExistMisMatch_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:24.196006+0000 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 4 -> 3 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe682695a0 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.195+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 6 0x7f9dfddfde80 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 6 0x7f9dfddfde80 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 make_request target is 0x10000000011.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 lat 0.000671 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 _do_lookup res is 0 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 _lookup have 0x1.head["AlternateNameRenameExistMisMatch_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 _lookup 0x1.head(faked_ino=0 nref=28 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_44790 = 0x10000000011.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.287 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000011.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000011.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 _lookup have 0x1.head["AlternateNameRenameExistMisMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 _do_lookup on #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_447902 => 2277440192 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2277440192 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 7 for mds.0 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b83f22560, mds: 0) 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.196716+0000 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:7 lookup #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.196702+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:7 lookup #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.196702+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f28cc0 con 0x560b8406bdf0 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe682695a0 2024-03-28T01:08:24.288 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:7 = 0 (0) Success) v1 ==== 849+0+0 (crc 0 0 0) 0x7f9de8042450 con 0x560b8406bdf0 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 7 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.196716+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 features 0xffffffffffffffff 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 update_snap_trace len 70 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 3 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 4 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x10000000012.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 0 2024-03-28T01:08:24.289 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsxLsXsxFsxcrwb -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000012.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXs 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.194233+0000 mtime 2024-03-28T01:08:24.194233+0000 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 dir hash is 2 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 34 , mds inodestat change_attr: 34 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 12 client.4725 insert_dentry_inode 'AlternateNameRenameExistMisMatch_447902' vino 0x10000000012.head in dir 0x1.head dn 0x560b83e53e70 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 12 client.4725 had dentry AlternateNameRenameExistMisMatch_447902 with correct vino 0x10000000012.head 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 15 client.4725 update_dentry_lease 0x1.head["AlternateNameRenameExistMisMatch_447902"] lease(mask 0 dur 0 altn foomismatch)) from 2024-03-28T01:08:24.196716+0000 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 4 -> 3 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe682695a0 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 7 0x7f9dfddfde80 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 7 0x7f9dfddfde80 2024-03-28T01:08:24.290 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 make_request target is 0x10000000012.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 lat 0.000609 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 _do_lookup res is 0 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 _lookup have 0x1.head["AlternateNameRenameExistMisMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 _lookup 0x1.head(faked_ino=0 nref=29 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_447902 = 0x10000000012.head(faked_ino=0 nref=15 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_447902 => 2277440192 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2277440192 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 choose_target_mds from dirfragtree hash 2024-03-28T01:08:24.291 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 8 for mds.0 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b83f27330, mds: 0) 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_inode_release enter(in:0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0), req:0x560b83f27330 mds:0, drop:Ls, unless:-, force:0) 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_inode_release enter(in:0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90), req:0x560b83f27330 mds:0, drop:Lsx, unless:-, force:0) 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_dentry_release enter(dn:0x560b83e53e70) 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0), req:0x560b83f27330 mds:0, drop:XxFs, unless:Fx, force:1) 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_dentry_release enter(dn:0x7f9dc4003190) 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0), req:0x560b83f27330 mds:0, drop:LxXxFs, unless:Fx, force:1) 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.197388+0000 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:8 rename #0x1/AlternateNameRenameExistMisMatch_447902 (foo2) #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.197357+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:8 rename #0x1/AlternateNameRenameExistMisMatch_447902 (foo2) #0x1/AlternateNameRenameExistMisMatch_44790 2024-03-28T01:08:24.197357+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f28cc0 con 0x560b8406bdf0 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe682697f0 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.292 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.196+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(revoke ino 0x10000000011 1 seq 3 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.186574+0000 ctime 2024-03-28T01:08:24.186574+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de800a040 con 0x560b8406bdf0 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 5 client.4725 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 mds.0 seq now 1 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 5 client.4725 handle_cap_grant on in 0x10000000011 mds.0 seq 3 caps now pAsXsFsxcrwb was pAsxXsxFsxcrwb 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) pAsxXsxFsxcrwb ctime 2024-03-28T01:08:24.186574+0000 mtime 2024-03-28T01:08:24.186574+0000 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 revocation of AxXx 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 check_caps on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) wanted Fxcb used Fcb issued pAsXsFsxcrwb revoking AxXx flags=1 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsxFsxcrwb revoking AxXx 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 completed revocation of AxXx 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 mark_caps_flushing Fw 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 mark_caps_clean 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 send_cap 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) mds.0 seq 3 used Fcb want Fxcb flush Fw retain pAsLsxXsFsxcrwbl held pAsxXsxFsxcrwb revoking AxXx dropping - 2024-03-28T01:08:24.293 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 15 client.4725 auth cap, requesting max_size 3 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000011 1 seq 3 tid 2 caps=pAsXsFsxcrwb dirty=Fw wanted=Fxcb follows 1 size 3/3 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.193698+0000 ctime 2024-03-28T01:08:24.193698+0000 change_attr 1) v12 -- 0x7f9dc400c170 con 0x560b8406bdf0 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_caps(revoke ino 0x10000000012 1 seq 3 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.194233+0000 ctime 2024-03-28T01:08:24.194233+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de80427b0 con 0x560b8406bdf0 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 mds.0 seq now 2 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 5 client.4725 handle_cap_grant on in 0x10000000012 mds.0 seq 3 caps now pAsXsFsxcrwb was pAsxXsxFsxcrwb 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) pAsxXsxFsxcrwb ctime 2024-03-28T01:08:24.194233+0000 mtime 2024-03-28T01:08:24.194233+0000 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 revocation of AxXx 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 check_caps on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) wanted Fxcb used Fcb issued pAsXsFsxcrwb revoking AxXx flags=1 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 cap mds.0 issued pAsXsFsxcrwb implemented pAsxXsxFsxcrwb revoking AxXx 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 completed revocation of AxXx 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 mark_caps_flushing Fw 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 mark_caps_clean 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 send_cap 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) mds.0 seq 3 used Fcb want Fxcb flush Fw retain pAsLsxXsFsxcrwbl held pAsxXsxFsxcrwb revoking AxXx dropping - 2024-03-28T01:08:24.294 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 15 client.4725 auth cap, requesting max_size 3 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000012 1 seq 3 tid 3 caps=pAsXsFsxcrwb dirty=Fw wanted=Fxcb follows 1 size 3/3 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.195876+0000 ctime 2024-03-28T01:08:24.195876+0000 change_attr 1) v12 -- 0x7f9de800a040 con 0x560b8406bdf0 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.197+0000 7f9dfde00640 10 client.4725 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.198+0000 7f9e00600640 1 -- 172.21.15.33:0/2612231146 <== osd.7 v2:172.21.15.194:6824/2849955959 3 ==== osd_op_reply(3 10000000011.00000000 [write 0~3] v23'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83eff1f0 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.198+0000 7f9df7400640 10 client.4725 _flushed 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.198+0000 7f9df7400640 5 client.4725 put_cap_ref dropped last FILE_BUFFER ref on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.198+0000 7f9df7400640 5 client.4725 put_cap_ref dropped last FILE_CACHE ref on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.198+0000 7f9df7400640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 2 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.198+0000 7f9df7400640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9e00600640 1 -- 172.21.15.33:0/2612231146 <== osd.7 v2:172.21.15.194:6824/2849955959 4 ==== osd_op_reply(4 10000000012.00000000 [write 0~3] v23'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83eff1f0 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9df7400640 10 client.4725 _flushed 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9df7400640 5 client.4725 put_cap_ref dropped last FILE_BUFFER ref on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.295 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9df7400640 5 client.4725 put_cap_ref dropped last FILE_CACHE ref on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9df7400640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 2 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9df7400640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 12 ==== client_caps(grant ino 0x10000000012 1 seq 4 caps=pAsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 3/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:24.195876+0000 ctime 2024-03-28T01:08:24.195876+0000 change_attr 1) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de800a040 con 0x560b8406bdf0 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9dfde00640 10 client.4725 mds.0 seq now 3 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9dfde00640 5 client.4725 handle_cap_grant on in 0x10000000012 mds.0 seq 4 caps now pAsXsFsxcrwb was pAsXsFsxcrwb 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) pAsXsFsxcrwb ctime 2024-03-28T01:08:24.195876+0000 mtime 2024-03-28T01:08:24.195876+0000 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9dfde00640 10 client.4725 caps unchanged at pAsXsFsxcrwb 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.199+0000 7f9dfde00640 10 client.4725 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 13 ==== client_reply(???:8 = -22 (22) Invalid argument safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de8038450 con 0x560b8406bdf0 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 8 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.197388+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 10 client.4725 insert_trace -- no trace 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe682697f0 2024-03-28T01:08:24.296 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 8 0x7f9dfddfde80 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 8 0x7f9dfddfde80 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 lat 0.003878 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 rename result is -22 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 trim_cache size 2 max 16384 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 8 client.4725 _rename(#0x1/AlternateNameRenameExistMisMatch_44790, #0x1/AlternateNameRenameExistMisMatch_447902) = -22 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 walk: AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_44790"] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=30 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 path_walk cur=0x1.head(faked_ino=0 nref=31 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 path_walk AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.297 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 0 0x1.head(faked_ino=0 nref=31 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 (path is AlternateNameRenameExistMisMatch_447902) 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 may_lookup 0x1.head(faked_ino=0 nref=31 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 _getattr mask As issued=1 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 3 client.4725 may_lookup 0x7f9dc400afd0 = 0 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 _lookup have 0x1.head["AlternateNameRenameExistMisMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 _do_lookup on #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 choose_target_mds inode dir hash is 2 on AlternateNameRenameExistMisMatch_447902 => 2277440192 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 choose_target_mds 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) is_hash=1 hash=2277440192 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 mds is 0 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 send_request rebuilding request 9 for mds.0 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 encode_cap_releases enter (req: 0x560b83f28cc0, mds: 0) 2024-03-28T01:08:24.298 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 send_request set sent_stamp to 2024-03-28T01:08:24.201348+0000 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 10 client.4725 send_request client_request(unknown.0:9 lookup #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.201333+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:9 lookup #0x1/AlternateNameRenameExistMisMatch_447902 2024-03-28T01:08:24.201333+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f21a40 con 0x560b8406bdf0 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9e0324a480 20 client.4725 awaiting reply|forward|kick on 0x7ffe682695f0 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 14 ==== client_reply(???:5 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9de8038450 con 0x560b8406bdf0 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 5 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.194262+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 10 client.4725 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.200+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 15 ==== client_reply(???:9 = 0 (0) Success) v1 ==== 849+0+0 (crc 0 0 0) 0x7f9de8038450 con 0x560b8406bdf0 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 handle_client_reply got a reply. Safe:1 tid 9 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 insert_trace from 2024-03-28T01:08:24.201348+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 features 0xffffffffffffffff 2024-03-28T01:08:24.299 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 update_snap_trace len 70 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref was 3 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 get_snap_realm 0x1 0x7f9dc4008f50, nref now is 4 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 update_snap_trace snaprealm(0x1 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 hrm is_target=1 is_dentry=1 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) caps pAsLsXsFsxcrwb 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) pAsXsFsxcrwb ctime 2024-03-28T01:08:24.195876+0000 mtime 2024-03-28T01:08:24.195876+0000 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 1 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsXsFsxcrwb -> pAsLsXsFsxcrwb from mds.0 on 0x10000000012.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 12 client.4725 add_update_inode had 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) caps pAsLsXsFs 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 update_inode_file_time 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400afd0) pAsLsXs ctime 2024-03-28T01:08:24.194233+0000 mtime 2024-03-28T01:08:24.194233+0000 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 dir hash is 2 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 12 client.4725 add_update_inode client inode change_attr: 34 , mds inodestat change_attr: 34 2024-03-28T01:08:24.300 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 add_update_cap calling signal_caps_inode 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 12 client.4725 insert_dentry_inode 'AlternateNameRenameExistMisMatch_447902' vino 0x10000000012.head in dir 0x1.head dn 0x560b83e53e70 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 12 client.4725 had dentry AlternateNameRenameExistMisMatch_447902 with correct vino 0x10000000012.head 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 update_dentry_lease 0x1.head["AlternateNameRenameExistMisMatch_447902"] lease(mask 0 dur 0 altn foomismatch)) from 2024-03-28T01:08:24.201348+0000 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 4 -> 3 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 handle_client_reply signalling caller 0x7ffe682695f0 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 handle_client_reply awaiting kickback on tid 9 0x7f9dfddfde80 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 sendrecv kickback on tid 9 0x7f9dfddfde80 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 make_request target is 0x10000000012.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 lat 0.000633 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _do_lookup res is 0 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 _lookup have 0x1.head["AlternateNameRenameExistMisMatch_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.301 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _lookup 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) AlternateNameRenameExistMisMatch_447902 = 0x10000000012.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 2 client.4725 unmount 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameRenameExistMisMatch_447902"] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size was 2+3 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 trim_cache size 2 max 16384 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 trim_dentry unlinking dn AlternateNameRenameExistMisMatch_44790 in dir 0x1 2024-03-28T01:08:24.302 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 unlink dir 0x7f9dc400afd0 'AlternateNameRenameExistMisMatch_44790' dn 0x7f9dc4003190 inode 0x7f9dc400b5b0 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 unlink inode 0x7f9dc400b5b0 parents now 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 unlink removing 'AlternateNameRenameExistMisMatch_44790' dn 0x7f9dc4003190 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 trim_dentry unlinking dn AlternateNameRenameExistMisMatch_447902 in dir 0x1 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 unlink dir 0x7f9dc400afd0 'AlternateNameRenameExistMisMatch_447902' dn 0x560b83e53e70 inode 0x7f9dc400bb90 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 unlink inode 0x7f9dc400bb90 parents now 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 unlink removing 'AlternateNameRenameExistMisMatch_447902' dn 0x560b83e53e70 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 15 client.4725 close_dir dir 0x560b83fbef20 on 0x7f9dc400afd0 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:24.303 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 2 client.4725 unmounting 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=32 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _ll_drop_pins 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 _ll_put 0x7f9dc400afd0 0x1 1 -> 0 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=33 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=33 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 _release 0x10000000012.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _invalidate_inode_cache 0x10000000012.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _flush 0x10000000012.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 nothing to flush 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 _release 0x10000000011.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _invalidate_inode_cache 0x10000000011.head(faked_ino=0 nref=18 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.304 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _flush 0x10000000011.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 nothing to flush 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 _release 0x1.head(faked_ino=0 nref=34 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=34 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 _flush 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 nothing to flush 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 20 client.4725 put_inode on 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 1 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 flush_caps_sync 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 wait_sync_caps want 3 (last is 3, 2 total flushing) 2024-03-28T01:08:24.305 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:24.201+0000 7f9e0324a480 10 client.4725 waiting on mds.0 tid 2 (want 3) 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 20 client.4725 tick 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 10 client.4725 renew_caps() 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 15 client.4725 renew_caps requesting from mds.0 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 10 client.4725 renew_caps mds.0 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 2) v7 -- 0x7f9de40033d0 con 0x560b8406bdf0 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 20 client.4725 collect_and_send_metrics 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 20 client.4725 collect_and_send_global_metrics 2024-03-28T01:08:25.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_metrics [client_metric_type: READ_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: WRITE_LATENCY latency: 0.002659, avg_latency: 0.001329, sq_sum: 3367318716962, count=2][client_metric_type: METADATA_LATENCY latency: 0.013183, avg_latency: 0.001464, sq_sum: 19440898473989, count=9][client_metric_type: CAP_INFO cap_hits: 15 cap_misses: 5 num_caps: 0][client_metric_type: DENTRY_LEASE dlease_hits: 0 dlease_misses: 6 num_dentries: 0][client_metric_type: OPENED_FILES opened_files: 0 total_inodes: 3][client_metric_type: PINNED_ICAPS pinned_icaps: 3 total_inodes: 3][client_metric_type: OPENED_INODES opened_inodes: 0 total_inodes: 3][client_metric_type: READ_IO_SIZES total_ops: 0 total_size: 0][client_metric_type: WRITE_IO_SIZES total_ops: 2 total_size: 6] v1 -- 0x7f9de4002b20 con 0x560b8406bdf0 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 10 client.4725 _put_inode on 0x1.head(faked_ino=0 nref=35 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) n = 34 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 10 client.4725 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 15 client.4725 remove_cap last one, closing snaprealm 0x7f9dc4008f50 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 3 -> 2 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 10 client.4725 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=- has_dir_layout 0x7f9dc400afd0) 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 10 client.4725 _put_inode on 0x10000000011.head(faked_ino=0 nref=19 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 17 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 10 client.4725 _put_inode on 0x10000000012.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 21 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9df5600640 20 client.4725 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 16 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faa5e0 con 0x560b8406bdf0 2024-03-28T01:08:25.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9dfde00640 10 client.4725 handle_client_session client_session(renewcaps seq 2) v1 from mds.0 2024-03-28T01:08:25.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size was 0+2 2024-03-28T01:08:25.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9dfde00640 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:25.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:25.164+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size still 0+2 2024-03-28T01:08:26.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:26.164+0000 7f9df5600640 20 client.4725 tick 2024-03-28T01:08:26.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:26.164+0000 7f9df5600640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_cap_release(1) v2 -- 0x7f9de400d120 con 0x560b8406bdf0 2024-03-28T01:08:26.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:26.164+0000 7f9df5600640 20 client.4725 collect_and_send_metrics 2024-03-28T01:08:26.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:26.164+0000 7f9df5600640 20 client.4725 collect_and_send_global_metrics 2024-03-28T01:08:26.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:26.164+0000 7f9df5600640 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_metrics [client_metric_type: READ_LATENCY latency: 0.000000, avg_latency: 0.000000, sq_sum: 0, count=0][client_metric_type: WRITE_LATENCY latency: 0.002659, avg_latency: 0.001329, sq_sum: 3367318716962, count=2][client_metric_type: METADATA_LATENCY latency: 0.013183, avg_latency: 0.001464, sq_sum: 19440898473989, count=9][client_metric_type: CAP_INFO cap_hits: 15 cap_misses: 5 num_caps: 0][client_metric_type: DENTRY_LEASE dlease_hits: 0 dlease_misses: 6 num_dentries: 0][client_metric_type: OPENED_FILES opened_files: 0 total_inodes: 2][client_metric_type: PINNED_ICAPS pinned_icaps: 2 total_inodes: 2][client_metric_type: OPENED_INODES opened_inodes: 0 total_inodes: 2][client_metric_type: READ_IO_SIZES total_ops: 0 total_size: 0][client_metric_type: WRITE_IO_SIZES total_ops: 2 total_size: 6] v1 -- 0x7f9de4009f30 con 0x560b8406bdf0 2024-03-28T01:08:26.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:26.164+0000 7f9df5600640 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:26.167 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:26.164+0000 7f9df5600640 20 client.4725 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:27.091 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 17 ==== client_caps(flush_ack ino 0x10000000011 1 seq 3 tid 2 caps=pAsXsFsxcrwb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400c170 con 0x560b8406bdf0 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 mds.0 seq now 4 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 5 client.4725 handle_cap_flush_ack mds.0 cleaned Fw on 0x10000000011.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) with Fw 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 5 client.4725 flushing_caps Fw -> - 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 0x10000000011.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) !flushing 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000011.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size was 0+2 2024-03-28T01:08:27.092 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size still 0+2 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9e0324a480 10 client.4725 wait_sync_caps want 3 (last is 3, 1 total flushing) 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9e0324a480 10 client.4725 waiting on mds.0 tid 3 (want 3) 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 18 ==== client_caps(flush_ack ino 0x10000000012 1 seq 3 tid 3 caps=pAsXsFsxcrwb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9de8038bf0 con 0x560b8406bdf0 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 mds.0 seq now 5 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 5 client.4725 handle_cap_flush_ack mds.0 cleaned Fw on 0x10000000012.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) with Fw 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 5 client.4725 flushing_caps Fw -> - 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 0x10000000012.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) !flushing 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 20 client.4725 put_inode on 0x10000000012.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size was 0+2 2024-03-28T01:08:27.093 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size still 0+2 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9e0324a480 10 client.4725 wait_sync_caps want 3 (last is 3, 0 total flushing) 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9e0324a480 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9e0324a480 10 client.4725 _put_inode on 0x10000000011.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) n = 1 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.088+0000 7f9e0324a480 10 client.4725 remove_cap mds.0 on 0x10000000011.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=pAsXsFsxcrwb(0=pAsXsFsxcrwb) objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 15 client.4725 remove_cap last one, closing snaprealm 0x7f9dc4008f50 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 2 -> 1 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 10 client.4725 _put_inode deleting 0x10000000011.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.186574+0000 mtime=2024-03-28T01:08:24.193698+0000 ctime=2024-03-28T01:08:24.193698+0000 change_attr=1 caps=- objectset[0x10000000011 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400b5b0) 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 10 client.4725 _put_inode on 0x10000000012.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) n = 1 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 10 client.4725 remove_cap mds.0 on 0x10000000012.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 15 client.4725 remove_cap last one, closing snaprealm 0x7f9dc4008f50 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 20 client.4725 put_snap_realm 0x1 0x7f9dc4008f50 1 -> 0 2024-03-28T01:08:27.094 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 10 client.4725 _put_inode deleting 0x10000000012.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:24.194233+0000 mtime=2024-03-28T01:08:24.195876+0000 ctime=2024-03-28T01:08:24.195876+0000 change_attr=1 caps=- objectset[0x10000000012 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bb90) 2024-03-28T01:08:27.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 2 client.4725 _close_mds_session mds.0 seq 5 2024-03-28T01:08:27.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 5) v7 -- 0x560b83faadc0 con 0x560b8406bdf0 2024-03-28T01:08:27.095 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.089+0000 7f9e0324a480 2 client.4725 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:27.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 <== mds.0 v2:172.21.15.33:6834/2290842450 19 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b8406bdf0 2024-03-28T01:08:27.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 10 client.4725 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:27.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 5 client.4725 _closed_mds_session mds.0 seq 5 2024-03-28T01:08:27.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 1 -- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406bdf0 msgr2=0x560b83ee4360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:27.141 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b8406bdf0 0x560b83ee4360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 10 client.4725 remove_session_caps mds.0 2024-03-28T01:08:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 10 client.4725 kick_requests_closed for mds.0 2024-03-28T01:08:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size was 0+0 2024-03-28T01:08:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9dfde00640 10 client.4725 unmounting: trim pass, size still 0+0 2024-03-28T01:08:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9e0324a480 2 client.4725 unmounted. 2024-03-28T01:08:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.138+0000 7f9e0324a480 1 client.4725 shutdown 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.139+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83eff1f0 msgr2=0x560b84053c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.139+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x560b83eff1f0 0x560b84053c50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.139+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9970 msgr2=0x560b83fba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.139+0000 7f9e0324a480 1 --2- 172.21.15.33:0/2612231146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83fa9970 0x560b83fba320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9df001bd70 tx=0x560b83fbab90 comp rx=0 tx=0).stop 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.139+0000 7f9e00600640 1 -- 172.21.15.33:0/2612231146 reap_dead start 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.140+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 shutdown_connections 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.140+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 >> 172.21.15.33:0/2612231146 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:27.143 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.141+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 shutdown_connections 2024-03-28T01:08:27.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.141+0000 7f9e0324a480 1 -- 172.21.15.33:0/2612231146 wait complete. 2024-03-28T01:08:27.144 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.141+0000 7f9e0324a480 20 client.4725 trim_cache size 0 max 16384 2024-03-28T01:08:27.145 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] TestClient.AlternateNameRenameExistMisMatch (2987 ms) 2024-03-28T01:08:27.145 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] TestClient.AlternateNameLink 2024-03-28T01:08:27.145 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.143+0000 7f9e0324a480 1 -- start start 2024-03-28T01:08:27.146 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.144+0000 7f9e0324a480 1 Processor -- start 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.145+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9970 0x560b84059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.145+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f99a60 0x560b83f2ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.145+0000 7f9e0324a480 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83ee4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.145+0000 7f9e0324a480 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4009fb0 con 0x560b83f99a60 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.145+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b83f237c0 con 0x560b83fa9970 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.145+0000 7f9e0324a480 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8406b050 con 0x560b83fad220 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.146+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9970 0x560b84059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.146+0000 7f9e00600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83ee4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.146+0000 7f9dff200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f99a60 0x560b83f2ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.146+0000 7f9e00600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83ee4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:40126/0 (socket says 172.21.15.33:40126) 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.146+0000 7f9e00600640 1 -- 172.21.15.33:0/3600016486 learned_addr learned my addr 172.21.15.33:0/3600016486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:27.149 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.146+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9970 0x560b84059ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:57014/0 (socket says 172.21.15.33:57014) 2024-03-28T01:08:27.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.147+0000 7f9dff200640 1 -- 172.21.15.33:0/3600016486 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 msgr2=0x560b83ee4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:27.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.147+0000 7f9dff200640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b83fad220 0x560b83ee4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:27.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.147+0000 7f9dff200640 1 -- 172.21.15.33:0/3600016486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9970 msgr2=0x560b84059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:27.150 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.147+0000 7f9dff200640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b83fa9970 0x560b84059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:27.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.147+0000 7f9dff200640 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b83fbb5a0 con 0x560b83f99a60 2024-03-28T01:08:27.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.148+0000 7f9dff200640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f99a60 0x560b83f2ca90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9de802b880 tx=0x7f9de8070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c711322d57a8e4b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:27.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.148+0000 7f9dfca00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-03-28T01:08:27.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.148+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4009f30 con 0x560b83f99a60 2024-03-28T01:08:27.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.148+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9de804a040 con 0x560b83f99a60 2024-03-28T01:08:27.151 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.148+0000 7f9e0324a480 10 client.4728 Subscribing to map 'mdsmap' 2024-03-28T01:08:27.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.148+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,mdsmap=0+,monmap=2+}) v3 -- 0x560b84055130 con 0x560b83f99a60 2024-03-28T01:08:27.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.149+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de400d120 con 0x560b83f99a60 2024-03-28T01:08:27.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.149+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mdsmap(e 8) v2 ==== 1180+0+0 (secure 0 0 0) 0x7f9de8017040 con 0x560b83f99a60 2024-03-28T01:08:27.152 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.149+0000 7f9dfca00640 1 client.4728 handle_mds_map epoch 8 2024-03-28T01:08:27.153 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9df6000640 20 client.4728 tick 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9df6000640 10 client.4728 renew_caps() 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9df6000640 20 client.4728 collect_and_send_metrics 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9df6000640 20 client.4728 collect_and_send_global_metrics 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9df6000640 5 client.4728 collect_and_send_global_metrics: no session with rank=0 -- not sending metric 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9df6000640 20 client.4728 trim_cache size 0 max 16384 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9df6000640 20 client.4728 upkeep thread waiting interval 1.000000000s 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9e0324a480 20 client.4728 populate_metadata read hostname 'smithi033' 2024-03-28T01:08:27.154 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9e0324a480 10 client.4728 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:27.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9e0324a480 20 client.4728 mds is 0 2024-03-28T01:08:27.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9e0324a480 10 client.4728 _open_mds_session mds.0 2024-03-28T01:08:27.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053230 0x560b8404ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:27.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_open) v7 -- 0x560b83faadc0 con 0x560b84053230 2024-03-28T01:08:27.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.151+0000 7f9e0324a480 10 client.4728 waiting for session to mds.0 to open 2024-03-28T01:08:27.155 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.152+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053230 0x560b8404ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:27.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.154+0000 7f9dffc00640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053230 0x560b8404ede0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9c4b4e2bba60ec server_cookie=5bfb43b52a48d7ea in_seq=0 out_seq=0 2024-03-28T01:08:27.156 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.154+0000 7f9dfca00640 10 client.4728 ms_handle_connect on v2:172.21.15.33:6834/2290842450 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 1 ==== client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 ==== 113+0+0 (crc 0 0 0) 0x560b83faadc0 con 0x560b84053230 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9dfca00640 10 client.4728 handle_client_session client_session(open cap_auths [MDSCapAuth(readable=1, writeable=1)]) v7 from mds.0 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9dfca00640 10 client.4728 renew_caps mds.0 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_renewcaps seq 1) v7 -- 0x7f9dc400b1c0 con 0x560b84053230 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9dfca00640 10 client.4728 connect_mds_targets for mds.0 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 10 client.4728 did not get mds through better means, so chose random mds 0 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 20 client.4728 mds is 0 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 10 client.4728 send_request rebuilding request 1 for mds.0 2024-03-28T01:08:27.168 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 20 client.4728 encode_cap_releases enter (req: 0x560b8406bdf0, mds: 0) 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 20 client.4728 send_request set sent_stamp to 2024-03-28T01:08:27.164263+0000 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 10 client.4728 send_request client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:27.151901+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:1 getattr pAsLsXsFs #0x1 2024-03-28T01:08:27.151901+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83eff1f0 con 0x560b84053230 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9e0324a480 20 client.4728 awaiting reply|forward|kick on 0x7ffe68269b30 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (crc 0 0 0) 0x7f9dc400b1c0 con 0x560b84053230 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.163+0000 7f9dfca00640 10 client.4728 handle_client_session client_session(renewcaps seq 1) v1 from mds.0 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 3 ==== client_reply(???:1 = 0 (0) Success) v1 ==== 431+0+0 (crc 0 0 0) 0x7f9df0059070 con 0x560b84053230 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 handle_client_reply got a reply. Safe:1 tid 1 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 insert_trace from 2024-03-28T01:08:27.164263+0000 mds.0 is_target=1 is_dentry=0 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 features 0xffffffffffffffff 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 update_snap_trace len 70 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0, nref was 0 2024-03-28T01:08:27.169 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref now is 1 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 update_snap_trace snaprealm(0x1 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 invalidate_snaprealm_and_children snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 15 client.4728 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) self|parent updated 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 15 client.4728 snapc 1=[] 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 no new snap on snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x1 0x7f9dc400b3e0 2 -> 1 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 hrm is_target=1 is_dentry=0 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- 0x7f9dc400b4d0) - ctime 2024-03-28T01:08:24.194233+0000 mtime 2024-03-28T01:08:24.194233+0000 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 dir hash is 2 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 12 client.4728 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 34 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 12 client.4728 add_update_inode adding 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=- has_dir_layout 0x7f9dc400b4d0) caps pAsLsXsFs 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref was 1 2024-03-28T01:08:27.170 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref now is 2 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 15 client.4728 add_update_cap first one, opened snaprealm 0x7f9dc400b3e0 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 add_update_cap issued - -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=3 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 add_update_cap calling signal_caps_inode 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x1 0x7f9dc400b3e0 2 -> 1 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 handle_client_reply signalling caller 0x7ffe68269b30 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 handle_client_reply awaiting kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 sendrecv kickback on tid 1 0x7f9dfc9fde80 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 lat 0.000820 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 _ll_get 0x7f9dc400b4d0 0x1 -> 1 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 3 client.4728 openat enter(AlternateNameLink_44790) 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=6 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 path_walk cur=0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.171 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 path_walk AlternateNameLink_44790 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 0 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) AlternateNameLink_44790 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 (path is AlternateNameLink_44790) 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 may_lookup 0x1.head(faked_ino=0 nref=10 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 _getattr mask As issued=1 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 3 client.4728 may_lookup 0x7f9dc400b4d0 = 0 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 _do_lookup on #0x1/AlternateNameLink_44790 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 choose_target_mds inode dir hash is 2 on AlternateNameLink_44790 => 4049707808 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 choose_target_mds 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) is_hash=1 hash=4049707808 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 mds is 0 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 send_request rebuilding request 2 for mds.0 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 encode_cap_releases enter (req: 0x560b83eff1f0, mds: 0) 2024-03-28T01:08:27.172 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 send_request set sent_stamp to 2024-03-28T01:08:27.165161+0000 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 send_request client_request(unknown.0:2 lookup #0x1/AlternateNameLink_44790 2024-03-28T01:08:27.165145+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:2 lookup #0x1/AlternateNameLink_44790 2024-03-28T01:08:27.165145+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83f28c40 con 0x560b84053230 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 awaiting reply|forward|kick on 0x7ffe68269390 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 4 ==== client_reply(???:2 = -2 (2) No such file or directory) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df00b5040 con 0x560b84053230 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 handle_client_reply got a reply. Safe:1 tid 2 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 insert_trace from 2024-03-28T01:08:27.165161+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 10 client.4728 insert_trace -- no trace 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 handle_client_reply signalling caller 0x7ffe68269390 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 handle_client_reply awaiting kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 sendrecv kickback on tid 2 0x7f9dfc9fde80 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 lat 0.000353 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 _do_lookup res is -2 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 _lookup 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) AlternateNameLink_44790 = -2 2024-03-28T01:08:27.173 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=11 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 path_walk cur=0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 path_walk 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 may_create 0x1.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 _getattr mask As issued=1 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 3 client.4728 may_create 0x7f9dc400b4d0 = 0 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 8 client.4728 _create(0x1 AlternateNameLink_44790, 0777) 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 get_or_create 0x1.head(faked_ino=0 nref=15 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) name AlternateNameLink_44790 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 15 open_dir 0x560b84055210 on 0x7f9dc400b4d0 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 15 client.4728 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 'AlternateNameLink_44790' dn 0x1.head["AlternateNameLink_44790"] (new dn) 2024-03-28T01:08:27.174 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 choose_target_mds inode dir hash is 2 on AlternateNameLink_44790 => 4049707808 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 choose_target_mds 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) is_hash=1 hash=4049707808 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 mds is 0 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 send_request rebuilding request 3 for mds.0 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 encode_cap_releases enter (req: 0x560b83f28c40, mds: 0) 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 encode_dentry_release enter(dn:0x7f9dc400aee0) 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0), req:0x560b83f28c40 mds:0, drop:AxXxFs, unless:Fx, force:1) 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 send_request set sent_stamp to 2024-03-28T01:08:27.165589+0000 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 10 client.4728 send_request client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameLink_44790 (foo) 2024-03-28T01:08:27.165570+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:3 create owner_uid=0, owner_gid=0 #0x1/AlternateNameLink_44790 (foo) 2024-03-28T01:08:27.165570+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fa5bc0 con 0x560b84053230 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9e0324a480 20 client.4728 awaiting reply|forward|kick on 0x7ffe682696f0 2024-03-28T01:08:27.175 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.164+0000 7f9dfca00640 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 5 ==== client_reply(???:3 = 0 (0) Success safe) v1 ==== 847+0+0 (crc 0 0 0) 0x7f9df0054bd0 con 0x560b84053230 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 handle_client_reply got a reply. Safe:1 tid 3 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 insert_trace from 2024-03-28T01:08:27.165589+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 features 0xffffffffffffffff 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 update_snap_trace len 70 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref was 1 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref now is 2 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 update_snap_trace snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 hrm is_target=1 is_dentry=1 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x10000000013.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=0.000000 ctime=0.000000 change_attr=0 caps=- objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) - ctime 2024-03-28T01:08:27.165570+0000 mtime 2024-03-28T01:08:27.165570+0000 2024-03-28T01:08:27.182 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 12 client.4728 add_update_inode client inode change_attr: 0 , mds inodestat change_attr: 0 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 12 client.4728 add_update_inode adding 0x10000000013.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=- objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref was 2 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref now is 3 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 15 client.4728 add_update_cap first one, opened snaprealm 0x7f9dc400b3e0 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 add_update_cap issued - -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000013.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={} open={} mode=100777 size=0/0 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 add_update_cap calling signal_caps_inode 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 12 client.4728 add_update_inode had 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) caps pAsLsXsFs 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:24.194233+0000 ctime=2024-03-28T01:08:24.194233+0000 change_attr=34 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) pAsLsXs ctime 2024-03-28T01:08:27.165570+0000 mtime 2024-03-28T01:08:27.165570+0000 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 dir hash is 2 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 12 client.4728 add_update_inode client inode change_attr: 34 , mds inodestat change_attr: 35 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 add_update_cap issued pAsLsXs -> pAsLsXsFs from mds.0 on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 10 client.4728 add_update_cap calling signal_caps_inode 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:27.183 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 12 client.4728 insert_dentry_inode 'AlternateNameLink_44790' vino 0x10000000013.head in dir 0x1.head dn 0x7f9dc400aee0 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 15 client.4728 link dir 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 'AlternateNameLink_44790' to inode 0x7f9dc400bab0 dn 0x1.head["AlternateNameLink_44790"] (old dn) 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 link inode 0x7f9dc400bab0 parents now 0x1.head["AlternateNameLink_44790"] 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 15 client.4728 update_dentry_lease 0x1.head["AlternateNameLink_44790"] lease(mask 0 dur 0 altn foo)) from 2024-03-28T01:08:27.165589+0000 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x1 0x7f9dc400b3e0 3 -> 2 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 handle_client_reply signalling caller 0x7ffe682696f0 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9dfca00640 20 client.4728 handle_client_reply awaiting kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 20 client.4728 sendrecv kickback on tid 3 0x7f9dfc9fde80 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 10 client.4728 delegated_inos: [] 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 10 client.4728 make_request created ino 0x10000000013 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 20 client.4728 make_request target is 0x10000000013.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 20 client.4728 lat 0.003049 2024-03-28T01:08:27.184 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000013.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000013.head(faked_ino=0 nref=6 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 10 client.4728 _create_fh 0x10000000013 mode 2 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.167+0000 7f9e0324a480 20 client.4728 trim_cache size 1 max 16384 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 8 client.4728 create(#0x1/AlternateNameLink_44790, 0100777 layout 0 0 0) = 0 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 3 client.4728 openat exit(AlternateNameLink_44790) 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 10 client.4728 get_quota_root realm 0x1 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 10 client.4728 get_quota_root 0x10000000013.head -> 0x1.head 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 10 client.4728 cur file size is 0 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f2b100 0x560b84053af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:27.185 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- osd_op(unknown.0.0:1 3.22 3:44834937:::10000000013.00000000:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83faa550 con 0x560b83f2b100 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- osd_op(unknown.0.0:2 3.22 3:44834937:::10000000013.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83fb3f10 con 0x560b83f2b100 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9dfca00640 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=16 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.168+0000 7f9e00600640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f2b100 0x560b84053af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.169+0000 7f9dfca00640 10 client.4728 ms_handle_connect on v2:172.21.15.33:6808/1101919847 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.169+0000 7f9e00600640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f2b100 0x560b84053af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=83a345f199ff1ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.169+0000 7f9e00600640 1 -- 172.21.15.33:0/3600016486 <== osd.2 v2:172.21.15.33:6808/1101919847 1 ==== osd_op_reply(1 10000000013.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83f2b100 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e00600640 1 -- 172.21.15.33:0/3600016486 <== osd.2 v2:172.21.15.33:6808/1101919847 2 ==== osd_op_reply(2 10000000013.00000000 [create] v23'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83f2b100 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 wanted_max_size 0 -> 3 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 get_caps 0x10000000013.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) have pAsxLsXsxFsxcrwb need AsFw want Fb revoking - 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 clear_suid_sgid 0x10000000013.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0); UserPerm(uid: 0, gid: 0) defer 0 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 snaprealm snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 5 client.4728 get_cap_ref got first FILE_BUFFER ref on 0x10000000013.head(faked_ino=0 nref=8 ll_ref=0 cap_refs={4=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.186 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 5 client.4728 get_cap_ref got first FILE_CACHE ref on 0x10000000013.head(faked_ino=0 nref=9 ll_ref=0 cap_refs={4=0,1024=0,4096=1,8192=0} open={2=1} mode=100777 size=0/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000013.head(faked_ino=0 nref=10 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) - -> Fw 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 cap_delay_requeue on 0x10000000013.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=0 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 get_quota_root realm 0x1 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 get_quota_root 0x10000000013.head -> 0x1.head 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 7 client.4728 wrote to 3, extending file size 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 mark_caps_dirty 0x10000000013.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) Fw -> Fw 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 cap_delay_requeue on 0x10000000013.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=1,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 3 client.4728 write(10, "...", 3, 0) = 3 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 3 client.4728 close enter(10) 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 8 client.4728 _release_fh 0x560b83f274d0 mode 2 on 0x10000000013.head(faked_ino=0 nref=11 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=1} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 _flush 0x10000000013.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- osd_op(unknown.0.0:3 3.22 3:44834937:::10000000013.00000000:head [write 0~3 in=3b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v9 -- 0x560b83faa550 con 0x560b83f2b100 2024-03-28T01:08:27.187 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 check_caps on 0x10000000013.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) wanted Fxcb used Fcb issued pAsxLsXsxFsxcrwb revoking - flags=0 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 cap mds.0 issued pAsxLsXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking - 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 delaying cap release 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 cap_delay_requeue on 0x10000000013.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 _release_fh 0x560b83f274d0 on inode 0x10000000013.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) no async_err state 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=12 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 3 client.4728 close exit(10) 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 path_walk cur=0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 path_walk AlternateNameLink_44790 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 0 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) AlternateNameLink_44790 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 (path is AlternateNameLink_44790) 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 may_lookup 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 _getattr mask As issued=1 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 3 client.4728 may_lookup 0x7f9dc400b4d0 = 0 2024-03-28T01:08:27.188 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 _lookup have 0x1.head["AlternateNameLink_44790"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 _lookup 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) AlternateNameLink_44790 = 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=17 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 path_walk cur=0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 path_walk 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 may_hardlink 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 _getattr mask As issued=1 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 3 client.4728 may_hardlink 0x7f9dc400bab0 = 0 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 may_create 0x1.head(faked_ino=0 nref=18 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 _getattr mask As issued=1 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 3 client.4728 may_create 0x7f9dc400b4d0 = 0 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 8 client.4728 _link(0x10000000013 to 0x1 AlternateNameLink_447902 uid 0 gid 0) 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 break_deleg: breaking delegs on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 delegations_broken: delegations empty on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 get_or_create 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) name AlternateNameLink_447902 2024-03-28T01:08:27.189 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 15 client.4728 link dir 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 'AlternateNameLink_447902' dn 0x1.head["AlternateNameLink_447902"] (new dn) 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 choose_target_mds inode dir hash is 2 on AlternateNameLink_447902 => 2868772314 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 choose_target_mds 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0) is_hash=1 hash=2868772314 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 choose_target_mds from dirfragtree hash 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 mds is 0 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 send_request rebuilding request 4 for mds.0 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 encode_cap_releases enter (req: 0x560b83fa5bc0, mds: 0) 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 encode_inode_release enter(in:0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXsFs(0=pAsLsXsFs) has_dir_layout 0x7f9dc400b4d0), req:0x560b83fa5bc0 mds:0, drop:LxFs, unless:Fx, force:0) 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 send_request set sent_stamp to 2024-03-28T01:08:27.172111+0000 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 10 client.4728 send_request client_request(unknown.0:4 link #0x1/AlternateNameLink_447902 (foo2) #0x10000000013 2024-03-28T01:08:27.172062+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:4 link #0x1/AlternateNameLink_447902 (foo2) #0x10000000013 2024-03-28T01:08:27.172062+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb4320 con 0x560b84053230 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.171+0000 7f9e0324a480 20 client.4728 awaiting reply|forward|kick on 0x7ffe682698d0 2024-03-28T01:08:27.190 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 6 ==== client_caps(revoke ino 0x10000000013 1 seq 2 caps=pAsxXsxFsxcrwb dirty=- wanted=pAsxXsxFxwb follows 0 size 0/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:27.165570+0000 ctime 2024-03-28T01:08:27.165570+0000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081050 con 0x560b84053230 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 5 client.4728 set_cap_epoch_barrier epoch = 23 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 mds.0 seq now 1 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 5 client.4728 handle_cap_grant on in 0x10000000013 mds.0 seq 2 caps now pAsxXsxFsxcrwb was pAsxLsXsxFsxcrwb 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) pAsxLsXsxFsxcrwb ctime 2024-03-28T01:08:27.165570+0000 mtime 2024-03-28T01:08:27.165570+0000 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 revocation of Ls 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 check_caps on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) wanted Fxcb used Fcb issued pAsxXsxFsxcrwb revoking Ls flags=1 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 cap mds.0 issued pAsxXsxFsxcrwb implemented pAsxLsXsxFsxcrwb revoking Ls 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 completed revocation of Ls 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 mark_caps_flushing Fw 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 mark_caps_clean 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) dirty_caps=Fw flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 send_cap 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 3] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) mds.0 seq 2 used Fcb want Fxcb flush Fw retain pAsxLxXsxFsxcrwbl held pAsxLsXsxFsxcrwb revoking Ls dropping - 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 15 client.4728 auth cap, requesting max_size 3 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000013 1 seq 2 tid 2 caps=pAsxXsxFsxcrwb dirty=Fw wanted=Fxcb follows 1 size 3/3 ts 1/18446744073709551615 mtime 2024-03-28T01:08:27.171811+0000 ctime 2024-03-28T01:08:27.171811+0000 change_attr 1) v12 -- 0x7f9dc400c090 con 0x560b84053230 2024-03-28T01:08:27.191 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.172+0000 7f9dfca00640 10 client.4728 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9e00600640 1 -- 172.21.15.33:0/3600016486 <== osd.2 v2:172.21.15.33:6808/1101919847 3 ==== osd_op_reply(3 10000000013.00000000 [write 0~3] v23'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df8075080 con 0x560b83f2b100 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9df7400640 10 client.4728 _flushed 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=1,4096=0,8192=1} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9df7400640 5 client.4728 put_cap_ref dropped last FILE_BUFFER ref on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9df7400640 5 client.4728 put_cap_ref dropped last FILE_CACHE ref on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9df7400640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 2 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9df7400640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 7 ==== client_reply(???:4 = 0 (0) Success unsafe) v1 ==== 827+0+0 (crc 0 0 0) 0x7f9df004adc0 con 0x560b84053230 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 20 client.4728 handle_client_reply got a reply. Safe:0 tid 4 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 insert_trace from 2024-03-28T01:08:27.172111+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 features 0xffffffffffffffff 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 update_snap_trace len 70 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref was 2 2024-03-28T01:08:27.192 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref now is 3 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 hrm is_target=1 is_dentry=1 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 12 client.4728 add_update_inode had 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=1 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) caps pAsxLsXsxFsxcrwb 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.171811+0000 change_attr=1 caps=pAsxXsxFsxcrwb(0=pAsxXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) pAsxXsxFsxcrwb ctime 2024-03-28T01:08:27.172062+0000 mtime 2024-03-28T01:08:27.171811+0000 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 12 client.4728 add_update_inode client inode change_attr: 1 , mds inodestat change_attr: 2 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 add_update_cap issued pAsxXsxFsxcrwb -> pAsxLsXsxFsxcrwb from mds.0 on 0x10000000013.head(faked_ino=0 nref=13 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"] 0x7f9dc400bab0) 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 add_update_cap calling signal_caps_inode 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 12 client.4728 add_update_inode had 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) caps pAsLsXs 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.173+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.165570+0000 ctime=2024-03-28T01:08:27.165570+0000 change_attr=35 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) pAsLsXs ctime 2024-03-28T01:08:27.172062+0000 mtime 2024-03-28T01:08:27.172062+0000 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 dir hash is 2 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 add_update_inode client inode change_attr: 35 , mds inodestat change_attr: 36 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:27.193 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 insert_dentry_inode 'AlternateNameLink_447902' vino 0x10000000013.head in dir 0x1.head dn 0x560b83e2a0d0 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 15 client.4728 link dir 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 'AlternateNameLink_447902' to inode 0x7f9dc400bab0 dn 0x1.head["AlternateNameLink_447902"] (old dn) 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 link inode 0x7f9dc400bab0 parents now 0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=16 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 15 client.4728 update_dentry_lease 0x1.head["AlternateNameLink_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:27.172111+0000 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x1 0x7f9dc400b3e0 3 -> 2 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 handle_client_reply signalling caller 0x7ffe682698d0 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 handle_client_reply awaiting kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 sendrecv kickback on tid 4 0x7f9dfc9fde80 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 lat 0.002632 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 link result is 0 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 trim_cache size 2 max 16384 2024-03-28T01:08:27.194 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 8 client.4728 link(#0x10000000013, #0x1/AlternateNameLink_447902) = 0 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=19 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 8 ==== client_caps(revoke ino 0x10000000013 1 seq 4 caps=pAsLsXsFsxcrwb dirty=- wanted=Fxcb follows 0 size 3/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:27.171811+0000 ctime 2024-03-28T01:08:27.172062+0000 change_attr 2) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9dc400c090 con 0x560b84053230 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 walk: AlternateNameLink_447902 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 path_walk cur=0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 path_walk AlternateNameLink_447902 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 0 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) AlternateNameLink_447902 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 (path is AlternateNameLink_447902) 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 may_lookup 0x1.head(faked_ino=0 nref=20 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0); UserPerm(uid: 0, gid: 0) 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 _getattr mask As issued=1 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 3 client.4728 may_lookup 0x7f9dc400b4d0 = 0 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 _lookup have 0x1.head["AlternateNameLink_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 _do_lookup on #0x1/AlternateNameLink_447902 2024-03-28T01:08:27.195 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 choose_target_mds starting with req->inode 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 choose_target_mds inode dir hash is 2 on AlternateNameLink_447902 => 2868772314 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 choose_target_mds 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) is_hash=1 hash=2868772314 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 choose_target_mds from caps on inode 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 mds is 0 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 send_request rebuilding request 5 for mds.0 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 encode_cap_releases enter (req: 0x560b83fb4320, mds: 0) 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 send_request set sent_stamp to 2024-03-28T01:08:27.174849+0000 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 send_request client_request(unknown.0:5 lookup #0x1/AlternateNameLink_447902 2024-03-28T01:08:27.174828+0000 caller_uid=0, caller_gid=0{}) v6 to mds.0 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_request(unknown.0:5 lookup #0x1/AlternateNameLink_447902 2024-03-28T01:08:27.174828+0000 caller_uid=0, caller_gid=0{}) v6 -- 0x560b83fb4780 con 0x560b84053230 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 awaiting reply|forward|kick on 0x7ffe68269650 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 mds.0 seq now 2 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 5 client.4728 handle_cap_grant on in 0x10000000013 mds.0 seq 4 caps now pAsLsXsFsxcrwb was pAsxLsXsxFsxcrwb 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x10000000013.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsxLsXsxFsxcrwb(0=pAsxLsXsxFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) pAsxLsXsxFsxcrwb ctime 2024-03-28T01:08:27.172062+0000 mtime 2024-03-28T01:08:27.171811+0000 2024-03-28T01:08:27.196 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 revocation of AxXx 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 check_caps on 0x10000000013.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) wanted - used Fc issued pAsLsXsFsxcrwb revoking AxXx flags=1 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 cap mds.0 issued pAsLsXsFsxcrwb implemented pAsxLsXsxFsxcrwb revoking AxXx 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 completed revocation of AxXx 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 send_cap 0x10000000013.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) mds.0 seq 4 used Fc want - flush - retain pAsLsXsFsc held pAsxLsXsxFsxcrwb revoking AxXx dropping Fxrwb 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 15 client.4728 auth cap, reset requested_max_size due to not wanting any file write cap 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_caps(update ino 0x10000000013 1 seq 4 caps=pAsLsXsFsc dirty=- wanted=- follows 0 size 3/4194304 ts 1/18446744073709551615 mtime 2024-03-28T01:08:27.171811+0000 ctime 2024-03-28T01:08:27.172062+0000 change_attr 2) v12 -- 0x7f9df0081050 con 0x560b84053230 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 handle_cap_grant calling signal_caps_inode 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el9/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/test/client/alternate_name.cc:193: Failure 2024-03-28T01:08:27.197 INFO:tasks.workunit.client.0.smithi033.stdout:Expected equality of these values: 2024-03-28T01:08:27.198 INFO:tasks.workunit.client.0.smithi033.stdout: wdr.alternate_name 2024-03-28T01:08:27.198 INFO:tasks.workunit.client.0.smithi033.stdout: Which is: "" 2024-03-28T01:08:27.198 INFO:tasks.workunit.client.0.smithi033.stdout: altname2 2024-03-28T01:08:27.198 INFO:tasks.workunit.client.0.smithi033.stdout: Which is: "foo2" 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 9 ==== client_reply(???:5 = 0 (0) Success) v1 ==== 827+0+0 (crc 0 0 0) 0x7f9df0063d20 con 0x560b84053230 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 handle_client_reply got a reply. Safe:1 tid 5 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 insert_trace from 2024-03-28T01:08:27.174849+0000 mds.0 is_target=1 is_dentry=1 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 features 0xffffffffffffffff 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 update_snap_trace len 70 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref was 2 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x1 0x7f9dc400b3e0, nref now is 3 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 update_snap_trace snaprealm(0x1 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) seq 1 <= 1 and same parent, SKIPPING 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 hrm is_target=1 is_dentry=1 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 add_update_inode had 0x10000000013.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsc(0=pAsLsXsFsc) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) caps pAsLsXsFsxcrwb 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x10000000013.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsc(0=pAsLsXsFsc) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) pAsLsXsFscw ctime 2024-03-28T01:08:27.172062+0000 mtime 2024-03-28T01:08:27.171811+0000 2024-03-28T01:08:27.203 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 add_update_inode client inode change_attr: 2 , mds inodestat change_attr: 2 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 add_update_cap issued pAsLsXsFsc -> pAsLsXsFsxcrwb from mds.0 on 0x10000000013.head(faked_ino=0 nref=17 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 add_update_cap calling signal_caps_inode 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 add_update_inode had 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) caps pAsLsXs 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 update_inode_file_time 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) pAsLsXs ctime 2024-03-28T01:08:27.172062+0000 mtime 2024-03-28T01:08:27.172062+0000 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 dir hash is 2 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 add_update_inode client inode change_attr: 36 , mds inodestat change_attr: 36 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 10 client.4728 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 got dirfrag map for 0x1 frag * to mds 0 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 insert_dentry_inode 'AlternateNameLink_447902' vino 0x10000000013.head in dir 0x1.head dn 0x560b83e2a0d0 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 12 client.4728 had dentry AlternateNameLink_447902 with correct vino 0x10000000013.head 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 15 client.4728 update_dentry_lease 0x1.head["AlternateNameLink_447902"] lease(mask 0 dur 0 altn foo2)) from 2024-03-28T01:08:27.174849+0000 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x1 0x7f9dc400b3e0 3 -> 2 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 handle_client_reply signalling caller 0x7ffe68269650 2024-03-28T01:08:27.204 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 handle_client_reply awaiting kickback on tid 5 0x7f9dfc9fde80 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 sendrecv kickback on tid 5 0x7f9dfc9fde80 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 make_request target is 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 lat 0.000693 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 _do_lookup res is 0 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 _lookup have 0x1.head["AlternateNameLink_447902"] from mds.-1 ttl 0.000000 seq 0 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 _lookup 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) AlternateNameLink_447902 = 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 2 client.4728 unmount 2024-03-28T01:08:27.205 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 2 client.4728 unmounting 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_flushmdlog) v7 -- 0x560b83f1d920 con 0x560b84053230 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.174+0000 7f9e0324a480 10 client.4728 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 10 ==== client_caps(flush_ack ino 0x10000000013 1 seq 2 tid 2 caps=pAsxXsxFsxcrwb dirty=Fw wanted=- follows 0 size 0/0 mtime 0.000000 ctime 0.000000 change_attr 0) v12 ==== 260+0+0 (crc 0 0 0) 0x7f9df0081050 con 0x560b84053230 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 mds.0 seq now 3 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 5 client.4728 handle_cap_flush_ack mds.0 cleaned Fw on 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) flushing_caps=Fw objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) with Fw 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 5 client.4728 flushing_caps Fw -> - 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) !flushing 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 handle_cap_flush_ack calling signal_caps_inode 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=20 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_44790"], 0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 unmounting: trim pass, size was 2+2 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 trim_cache size 2 max 16384 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 15 client.4728 trim_dentry unlinking dn AlternateNameLink_44790 in dir 0x1 2024-03-28T01:08:27.206 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 15 client.4728 unlink dir 0x7f9dc400b4d0 'AlternateNameLink_44790' dn 0x7f9dc400aee0 inode 0x7f9dc400bab0 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 unlink inode 0x7f9dc400bab0 parents now 0x1.head["AlternateNameLink_447902"] 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 15 client.4728 unlink removing 'AlternateNameLink_44790' dn 0x7f9dc400aee0 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9e0324a480 10 client.4728 waiting for write request '4' to complete, currently there are 1 outstanding read/write requests 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 11 ==== client_snap(split split=0x10000000013 tracelen=140) v1 ==== 172+0+0 (crc 0 0 0) 0x7f9df006c480 con 0x560b84053230 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 handle_snap client_snap(split split=0x10000000013 tracelen=140) v1 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 mds.0 seq now 4 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x10000000013 0, nref was 0 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x10000000013 0x7f9dc400aee0, nref now is 1 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 splitting off snaprealm(0x10000000013 nref=1 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 moving 0x10000000013.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) from snaprealm(0x1 nref=2 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=2024-03-28T01:06:22.699088+0000 change_attr=0) 2024-03-28T01:08:27.207 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x1 0x7f9dc400b3e0 2 -> 1 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 update_snap_trace len 140 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x10000000013 0x7f9dc400aee0, nref was 1 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x10000000013 0x7f9dc400aee0, nref now is 2 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 update_snap_trace snaprealm(0x10000000013 nref=2 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 adjust_realm_parent snaprealm(0x10000000013 nref=3 c=1 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) 0x0 -> 0x3 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x3 0, nref was 0 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x3 0x7f9dc4006000, nref now is 2 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 invalidate_snaprealm_and_children snaprealm(0x10000000013 nref=3 c=1 seq=1 parent=0x3 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 15 client.4728 update_snap_trace snaprealm(0x10000000013 nref=3 c=1 seq=1 parent=0x3 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) self|parent updated 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 15 client.4728 snapc 1=[] 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x3 0x7f9dc4006000, nref was 2 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 get_snap_realm 0x3 0x7f9dc4006000, nref now is 3 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 update_snap_trace snaprealm(0x3 nref=3 c=0 seq=0 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) seq 1 > 0 2024-03-28T01:08:27.208 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 invalidate_snaprealm_and_children snaprealm(0x3 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 invalidate_snaprealm_and_children snaprealm(0x10000000013 nref=3 c=1 seq=1 parent=0x3 my_snaps=[] cached_snapc=1=[] last_modified=0.000000 change_attr=0) 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 15 client.4728 update_snap_trace snaprealm(0x3 nref=4 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=0=[] last_modified=0.000000 change_attr=0) self|parent updated 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 15 client.4728 snapc 1=[] 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x3 0x7f9dc4006000 4 -> 3 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 no new snap on snaprealm(0x3 nref=3 c=0 seq=1 parent=0x0 my_snaps=[] cached_snapc=1=[] last_modified=0.000000 change_attr=0) 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x3 0x7f9dc4006000 3 -> 2 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 no new snap on snaprealm(0x10000000013 nref=3 c=1 seq=1 parent=0x3 my_snaps=[] cached_snapc=1=[] last_modified=0.000000 change_attr=0) 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x10000000013 0x7f9dc400aee0 3 -> 2 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x10000000013 0x7f9dc400aee0 2 -> 1 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 put_snap_realm 0x10000000013 0x7f9dc400aee0 2 -> 1 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 unmounting: trim pass, size was 1+2 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 20 client.4728 trim_cache size 1 max 16384 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.177+0000 7f9dfca00640 10 client.4728 unmounting: trim pass, size still 1+2 2024-03-28T01:08:27.209 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 12 ==== client_reply(???:4 = 0 (0) Success safe) v1 ==== 27+0+0 (crc 0 0 0) 0x7f9df006c670 con 0x560b84053230 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 handle_client_reply got a reply. Safe:1 tid 4 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 10 client.4728 insert_trace from 2024-03-28T01:08:27.172111+0000 mds.0 is_target=0 is_dentry=0 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 10 client.4728 insert_trace -- already got unsafe; ignoring 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=21 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] parents=0x1.head["AlternateNameLink_447902"] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 10 client.4728 unmounting: trim pass, size was 1+2 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 trim_cache size 1 max 16384 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 15 client.4728 trim_dentry unlinking dn AlternateNameLink_447902 in dir 0x1 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 15 client.4728 unlink dir 0x7f9dc400b4d0 'AlternateNameLink_447902' dn 0x560b83e2a0d0 inode 0x7f9dc400bab0 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 unlink inode 0x7f9dc400bab0 parents now 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 15 client.4728 unlink removing 'AlternateNameLink_447902' dn 0x560b83e2a0d0 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 15 client.4728 close_dir dir 0x560b84055210 on 0x7f9dc400b4d0 2024-03-28T01:08:27.210 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9dfca00640 10 client.4728 unmounting: trim pass, cache shrank, poking unmount() 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=21 ll_ref=1 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _ll_drop_pins 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 _ll_put 0x7f9dc400b4d0 0x1 1 -> 0 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=22 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 _release 0x10000000013.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bab0) 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _invalidate_inode_cache 0x10000000013.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 1 dirty_or_tx 0] 0x7f9dc400bab0) 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _flush 0x10000000013.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 nothing to flush 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.211 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 _release 0x1.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _invalidate_inode_cache 0x1.head(faked_ino=0 nref=23 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _flush 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 nothing to flush 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x10000000013.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) n = 1 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 1 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 flush_caps_sync 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 wait_sync_caps want 2 (last is 2, 0 total flushing) 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 trim_cache size 0 max 16384 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _put_inode on 0x1.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) n = 23 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 remove_cap mds.0 on 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=pAsLsXs(0=pAsLsXs) has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 15 client.4728 remove_cap last one, closing snaprealm 0x7f9dc400b3e0 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_snap_realm 0x1 0x7f9dc400b3e0 1 -> 0 2024-03-28T01:08:27.212 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _put_inode deleting 0x1.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={1024=0} open={} mode=40755 size=0/0 nlink=1 btime=2024-03-28T01:06:22.699088+0000 mtime=2024-03-28T01:08:27.172062+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=36 caps=- has_dir_layout 0x7f9dc400b4d0) 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _put_inode on 0x10000000013.head(faked_ino=0 nref=24 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) n = 23 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 remove_cap mds.0 on 0x10000000013.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=pAsLsXsFsxcrwb(0=pAsLsXsFsxcrwb) objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 15 client.4728 remove_cap last one, closing snaprealm 0x7f9dc400aee0 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_snap_realm 0x10000000013 0x7f9dc400aee0 1 -> 0 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 20 client.4728 put_snap_realm 0x3 0x7f9dc4006000 2 -> 1 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 10 client.4728 _put_inode deleting 0x10000000013.head(faked_ino=0 nref=1 ll_ref=0 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=0} mode=100777 size=3/4194304 nlink=2 btime=2024-03-28T01:08:27.165570+0000 mtime=2024-03-28T01:08:27.171811+0000 ctime=2024-03-28T01:08:27.172062+0000 change_attr=2 caps=- objectset[0x10000000013 ts 0/0 objects 0 dirty_or_tx 0] 0x7f9dc400bab0) 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 2 client.4728 _close_mds_session mds.0 seq 4 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 --> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] -- client_session(request_close seq 4) v7 -- 0x560b83f2e3a0 con 0x560b84053230 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.178+0000 7f9e0324a480 2 client.4728 waiting for 1 mds session(s) to close (timeout: 30s) 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.180+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 <== mds.0 v2:172.21.15.33:6834/2290842450 13 ==== client_session(close) v1 ==== 28+0+0 (crc 0 0 0) 0x560b83f1d920 con 0x560b84053230 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.180+0000 7f9dfca00640 10 client.4728 handle_client_session client_session(close) v1 from mds.0 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.180+0000 7f9dfca00640 5 client.4728 _closed_mds_session mds.0 seq 4 2024-03-28T01:08:27.213 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.180+0000 7f9dfca00640 1 -- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053230 msgr2=0x560b8404ede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.180+0000 7f9dfca00640 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6834/2290842450,v1:172.21.15.33:6835/2290842450] conn(0x560b84053230 0x560b8404ede0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9dfca00640 10 client.4728 remove_session_caps mds.0 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9dfca00640 10 client.4728 kick_requests_closed for mds.0 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 20 client.4728 put_snap_realm 0x3 0x7f9dc4006000 1 -> 0 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 2 client.4728 unmounted. 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 client.4728 shutdown 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f2b100 msgr2=0x560b84053af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x560b83f2b100 0x560b84053af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f99a60 msgr2=0x560b83f2ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 --2- 172.21.15.33:0/3600016486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b83f99a60 0x560b83f2ca90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9de802b880 tx=0x7f9de8070000 comp rx=0 tx=0).stop 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e00600640 1 -- 172.21.15.33:0/3600016486 reap_dead start 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 shutdown_connections 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 >> 172.21.15.33:0/3600016486 conn(0x560b83f2d970 msgr2=0x560b8405a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:27.214 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 shutdown_connections 2024-03-28T01:08:27.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 1 -- 172.21.15.33:0/3600016486 wait complete. 2024-03-28T01:08:27.215 INFO:tasks.workunit.client.0.smithi033.stderr:2024-03-28T01:08:27.181+0000 7f9e0324a480 20 client.4728 trim_cache size 0 max 16384 2024-03-28T01:08:27.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[ FAILED ] TestClient.AlternateNameLink (39 ms) 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 25 tests from TestClient (13785 ms total) 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment tear-down 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] 25 tests from 1 test suite ran. (13785 ms total) 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[ PASSED ] 23 tests. 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[ FAILED ] 2 tests, listed below: 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[ FAILED ] TestClient.AlternateNameSymlink 2024-03-28T01:08:27.216 INFO:tasks.workunit.client.0.smithi033.stdout:[ FAILED ] TestClient.AlternateNameLink 2024-03-28T01:08:27.217 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-03-28T01:08:27.217 INFO:tasks.workunit.client.0.smithi033.stdout: 2 FAILED TESTS 2024-03-28T01:08:27.217 INFO:tasks.workunit:Stopping ['client/test.sh'] on client.0... 2024-03-28T01:08:27.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-28T01:08:28.227 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test client/test.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/client/test.sh' 2024-03-28T01:08:28.451 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3b61430755c448adb3c195d7fdbac6b6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test client/test.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/client/test.sh' 2024-03-28T01:08:28.454 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-28T01:08:28.464 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-28T01:08:28.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:08:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.715+0000 7f1b9d800640 1 Processor -- start 2024-03-28T01:08:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.715+0000 7f1b9d800640 1 -- start start 2024-03-28T01:08:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.716+0000 7f1b9d800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f1b980b51d0 0x7f1b980b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.716+0000 7f1b9d800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b980b55b0 con 0x7f1b980b42f0 2024-03-28T01:08:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.716+0000 7f1b9d800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b980b9190 con 0x7f1b980b8d00 2024-03-28T01:08:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.716+0000 7f1b9d800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9814a820 con 0x7f1b980b51d0 2024-03-28T01:08:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b9ce00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1b980b42f0 0x7f1b98151b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43634/0 (socket says 172.21.15.33:43634) 2024-03-28T01:08:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b9ce00640 1 -- 172.21.15.33:0/1525326440 learned_addr learned my addr 172.21.15.33:0/1525326440 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b97e00640 1 --2- 172.21.15.33:0/1525326440 >> v2:172.21.15.194:3300/0 conn(0x7f1b980b51d0 0x7f1b980b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124132230 0 0) 0x7f1b980b55b0 con 0x7f1b980b42f0 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b7c003520 con 0x7f1b980b42f0 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2580242232 0 0) 0x7f1b7c003520 con 0x7f1b980b42f0 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 >> v2:172.21.15.194:3300/0 conn(0x7f1b980b51d0 msgr2=0x7f1b980b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b96a00640 1 --2- 172.21.15.33:0/1525326440 >> v2:172.21.15.194:3300/0 conn(0x7f1b980b51d0 0x7f1b980b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.717+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 >> v1:172.21.15.194:6789/0 conn(0x7f1b980b8d00 legacy=0x7f1b9814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980b9800 con 0x7f1b980b42f0 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f1b90001670 con 0x7f1b980b42f0 2024-03-28T01:08:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 >> v1:172.21.15.33:6789/0 conn(0x7f1b980b42f0 legacy=0x7f1b98151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c004800 0x7f1b7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0051c0 0x7f1b7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c00c030 con 0x7f1b7c0039b0 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c00c1e0 con 0x7f1b7c004800 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c00c3e0 con 0x7f1b7c0051c0 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b97e00640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:28.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b97400640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c004800 0x7f1b7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b97e00640 1 -- 172.21.15.33:0/1525326440 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0051c0 msgr2=0x7f1b7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b97e00640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0051c0 0x7f1b7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b97e00640 1 -- 172.21.15.33:0/1525326440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c004800 msgr2=0x7f1b7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b97e00640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c004800 0x7f1b7c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b97e00640 1 -- 172.21.15.33:0/1525326440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c003520 con 0x7f1b7c0039b0 2024-03-28T01:08:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.718+0000 7f1b9ce00640 1 -- 172.21.15.33:0/1525326440 reap_dead start 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.719+0000 7f1b97e00640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009ac0 tx=0x7f1b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73931a69af44cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80039070 con 0x7f1b7c0039b0 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8002c6b0 con 0x7f1b7c0039b0 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b96a00640 1 -- 172.21.15.33:0/1525326440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80009690 con 0x7f1b7c0039b0 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b9d800640 1 -- 172.21.15.33:0/1525326440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b7c0039b0 msgr2=0x7f1b7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b9d800640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009ac0 tx=0x7f1b7c005830 comp rx=0 tx=0).stop 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b9d800640 1 -- 172.21.15.33:0/1525326440 shutdown_connections 2024-03-28T01:08:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b9d800640 1 --2- 172.21.15.33:0/1525326440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b9d800640 1 -- 172.21.15.33:0/1525326440 >> 172.21.15.33:0/1525326440 conn(0x7f1b98072940 msgr2=0x7f1b98072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b9d800640 1 -- 172.21.15.33:0/1525326440 shutdown_connections 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.720+0000 7f1b9d800640 1 -- 172.21.15.33:0/1525326440 wait complete. 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.722+0000 7f1b9d800640 1 Processor -- start 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.722+0000 7f1b9d800640 1 -- start start 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.723+0000 7f1b9d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0039b0 0x7f1b981dce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.723+0000 7f1b9d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b981d2960 0x7f1b981dd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9d800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b981e0100 0x7f1b981e0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9d800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980b6b80 con 0x7f1b981e0100 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9d800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980b39c0 con 0x7f1b7c0039b0 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9d800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980b6d00 con 0x7f1b981d2960 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b981e0100 0x7f1b981e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b981e0100 0x7f1b981e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32776/0 (socket says 172.21.15.33:32776) 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 -- 172.21.15.33:0/3516702858 learned_addr learned my addr 172.21.15.33:0/3516702858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b97e00640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0039b0 0x7f1b981dce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b97400640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b981d2960 0x7f1b981dd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 -- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b981d2960 msgr2=0x7f1b981dd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b981d2960 0x7f1b981dd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 -- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0039b0 msgr2=0x7f1b981dce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0039b0 0x7f1b981dce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.724+0000 7f1b9ce00640 1 -- 172.21.15.33:0/3516702858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b981d0f40 con 0x7f1b981e0100 2024-03-28T01:08:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.725+0000 7f1b9ce00640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b981e0100 0x7f1b981e0580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b980b59b0 tx=0x7f1b9000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.725+0000 7f1b8fe00640 1 -- 172.21.15.33:0/3516702858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90004bf0 con 0x7f1b981e0100 2024-03-28T01:08:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.725+0000 7f1b8fe00640 1 -- 172.21.15.33:0/3516702858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b90004db0 con 0x7f1b981e0100 2024-03-28T01:08:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.725+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b980a9b20 con 0x7f1b981e0100 2024-03-28T01:08:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.725+0000 7f1b8fe00640 1 -- 172.21.15.33:0/3516702858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b900073b0 con 0x7f1b981e0100 2024-03-28T01:08:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.725+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b981dd8d0 con 0x7f1b981e0100 2024-03-28T01:08:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.726+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98075310 con 0x7f1b981e0100 2024-03-28T01:08:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.727+0000 7f1b8fe00640 1 -- 172.21.15.33:0/3516702858 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1b9001d3f0 con 0x7f1b981e0100 2024-03-28T01:08:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.728+0000 7f1b8fe00640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1b7406bbd0 0x7f1b7406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.728+0000 7f1b8fe00640 1 -- 172.21.15.33:0/3516702858 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f1b9008a6b0 con 0x7f1b981e0100 2024-03-28T01:08:28.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.728+0000 7f1b97e00640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1b7406bbd0 0x7f1b7406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:28.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.729+0000 7f1b97e00640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1b7406bbd0 0x7f1b7406e090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b80036000 tx=0x7f1b80030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:28.733+0000 7f1b8fe00640 1 -- 172.21.15.33:0/3516702858 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f1b90059170 con 0x7f1b981e0100 2024-03-28T01:08:29.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.099+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b98145cd0 con 0x7f1b7406bbd0 2024-03-28T01:08:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.113+0000 7f1b8fe00640 1 -- 172.21.15.33:0/3516702858 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252146 (secure 0 0 0) 0x7f1b80082910 con 0x7f1b7406bbd0 2024-03-28T01:08:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:08:29.120 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:08:29.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1b7406bbd0 msgr2=0x7f1b7406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1b7406bbd0 0x7f1b7406e090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b80036000 tx=0x7f1b80030000 comp rx=0 tx=0).stop 2024-03-28T01:08:29.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b981e0100 msgr2=0x7f1b981e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b981e0100 0x7f1b981e0580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b980b59b0 tx=0x7f1b9000c960 comp rx=0 tx=0).stop 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 shutdown_connections 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f1b7406bbd0 0x7f1b7406e090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b981e0100 0x7f1b981e0580 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b981d2960 0x7f1b981dd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 --2- 172.21.15.33:0/3516702858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0039b0 0x7f1b981dce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 >> 172.21.15.33:0/3516702858 conn(0x7f1b98072940 msgr2=0x7f1b980bd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 shutdown_connections 2024-03-28T01:08:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.118+0000 7f1b9d800640 1 -- 172.21.15.33:0/3516702858 wait complete. 2024-03-28T01:08:29.148 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-03-28T01:08:28.428123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":721207,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":125,"num_read_kb":109,"num_write":345,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":813,"ondisk_log_size":813,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219552,"kb_used_data":3424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755168,"statfs":{"total":773094113280,"available":772869292032,"internally_reserved":0,"allocated":3506176,"data_stored":2157752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117370,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003984"},"pg_stats":[{"pgid":"3.3f","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182684+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:08:24.182684+0000","last_peered":"2024-03-28T01:08:24.182684+0000","last_clean":"2024-03-28T01:08:24.182684+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:08:24.182684+0000","last_undegraded":"2024-03-28T01:08:24.182684+0000","last_fullsized":"2024-03-28T01:08:24.182684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'4","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.170596+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:08:24.170596+0000","last_peered":"2024-03-28T01:08:24.170596+0000","last_clean":"2024-03-28T01:08:24.170596+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:08:24.170596+0000","last_undegraded":"2024-03-28T01:08:24.170596+0000","last_fullsized":"2024-03-28T01:08:24.170596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.170497+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:08:24.170497+0000","last_peered":"2024-03-28T01:08:24.170497+0000","last_clean":"2024-03-28T01:08:24.170497+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:08:24.170497+0000","last_undegraded":"2024-03-28T01:08:24.170497+0000","last_fullsized":"2024-03-28T01:08:24.170497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184333+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:08:24.184333+0000","last_peered":"2024-03-28T01:08:24.184333+0000","last_clean":"2024-03-28T01:08:24.184333+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:08:24.184333+0000","last_undegraded":"2024-03-28T01:08:24.184333+0000","last_fullsized":"2024-03-28T01:08:24.184333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183192+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:08:24.183192+0000","last_peered":"2024-03-28T01:08:24.183192+0000","last_clean":"2024-03-28T01:08:24.183192+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:08:24.183192+0000","last_undegraded":"2024-03-28T01:08:24.183192+0000","last_fullsized":"2024-03-28T01:08:24.183192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172793+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:08:24.172793+0000","last_peered":"2024-03-28T01:08:24.172793+0000","last_clean":"2024-03-28T01:08:24.172793+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:08:24.172793+0000","last_undegraded":"2024-03-28T01:08:24.172793+0000","last_fullsized":"2024-03-28T01:08:24.172793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.166510+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:08:24.166510+0000","last_peered":"2024-03-28T01:08:24.166510+0000","last_clean":"2024-03-28T01:08:24.166510+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:08:24.166510+0000","last_undegraded":"2024-03-28T01:08:24.166510+0000","last_fullsized":"2024-03-28T01:08:24.166510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183817+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:08:24.183817+0000","last_peered":"2024-03-28T01:08:24.183817+0000","last_clean":"2024-03-28T01:08:24.183817+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:08:24.183817+0000","last_undegraded":"2024-03-28T01:08:24.183817+0000","last_fullsized":"2024-03-28T01:08:24.183817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.185365+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:08:24.185365+0000","last_peered":"2024-03-28T01:08:24.185365+0000","last_clean":"2024-03-28T01:08:24.185365+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:08:24.185365+0000","last_undegraded":"2024-03-28T01:08:24.185365+0000","last_fullsized":"2024-03-28T01:08:24.185365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'17","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184340+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:08:24.184340+0000","last_peered":"2024-03-28T01:08:24.184340+0000","last_clean":"2024-03-28T01:08:24.184340+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:08:24.184340+0000","last_undegraded":"2024-03-28T01:08:24.184340+0000","last_fullsized":"2024-03-28T01:08:24.184340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183682+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:08:24.183682+0000","last_peered":"2024-03-28T01:08:24.183682+0000","last_clean":"2024-03-28T01:08:24.183682+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:08:24.183682+0000","last_undegraded":"2024-03-28T01:08:24.183682+0000","last_fullsized":"2024-03-28T01:08:24.183682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.161730+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:08:24.161730+0000","last_peered":"2024-03-28T01:08:24.161730+0000","last_clean":"2024-03-28T01:08:24.161730+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:08:24.161730+0000","last_undegraded":"2024-03-28T01:08:24.161730+0000","last_fullsized":"2024-03-28T01:08:24.161730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182627+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:08:24.182627+0000","last_peered":"2024-03-28T01:08:24.182627+0000","last_clean":"2024-03-28T01:08:24.182627+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:08:24.182627+0000","last_undegraded":"2024-03-28T01:08:24.182627+0000","last_fullsized":"2024-03-28T01:08:24.182627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172340+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:08:24.172340+0000","last_peered":"2024-03-28T01:08:24.172340+0000","last_clean":"2024-03-28T01:08:24.172340+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:08:24.172340+0000","last_undegraded":"2024-03-28T01:08:24.172340+0000","last_fullsized":"2024-03-28T01:08:24.172340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171787+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:08:24.171787+0000","last_peered":"2024-03-28T01:08:24.171787+0000","last_clean":"2024-03-28T01:08:24.171787+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:08:24.171787+0000","last_undegraded":"2024-03-28T01:08:24.171787+0000","last_fullsized":"2024-03-28T01:08:24.171787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.173992+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:08:24.173992+0000","last_peered":"2024-03-28T01:08:24.173992+0000","last_clean":"2024-03-28T01:08:24.173992+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:08:24.173992+0000","last_undegraded":"2024-03-28T01:08:24.173992+0000","last_fullsized":"2024-03-28T01:08:24.173992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.176879+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:08:24.176879+0000","last_peered":"2024-03-28T01:08:24.176879+0000","last_clean":"2024-03-28T01:08:24.176879+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:08:24.176879+0000","last_undegraded":"2024-03-28T01:08:24.176879+0000","last_fullsized":"2024-03-28T01:08:24.176879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179164+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:08:24.179164+0000","last_peered":"2024-03-28T01:08:24.179164+0000","last_clean":"2024-03-28T01:08:24.179164+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:08:24.179164+0000","last_undegraded":"2024-03-28T01:08:24.179164+0000","last_fullsized":"2024-03-28T01:08:24.179164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'7","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.169769+0000","last_change":"2024-03-28T01:07:03.939021+0000","last_active":"2024-03-28T01:08:24.169769+0000","last_peered":"2024-03-28T01:08:24.169769+0000","last_clean":"2024-03-28T01:08:24.169769+0000","last_became_active":"2024-03-28T01:07:03.937565+0000","last_became_peered":"2024-03-28T01:07:03.937565+0000","last_unstale":"2024-03-28T01:08:24.169769+0000","last_undegraded":"2024-03-28T01:08:24.169769+0000","last_fullsized":"2024-03-28T01:08:24.169769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:13:26.015917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181877+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:08:24.181877+0000","last_peered":"2024-03-28T01:08:24.181877+0000","last_clean":"2024-03-28T01:08:24.181877+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:08:24.181877+0000","last_undegraded":"2024-03-28T01:08:24.181877+0000","last_fullsized":"2024-03-28T01:08:24.181877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.178531+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:08:24.178531+0000","last_peered":"2024-03-28T01:08:24.178531+0000","last_clean":"2024-03-28T01:08:24.178531+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:08:24.178531+0000","last_undegraded":"2024-03-28T01:08:24.178531+0000","last_fullsized":"2024-03-28T01:08:24.178531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183967+0000","last_change":"2024-03-28T01:07:03.972650+0000","last_active":"2024-03-28T01:08:24.183967+0000","last_peered":"2024-03-28T01:08:24.183967+0000","last_clean":"2024-03-28T01:08:24.183967+0000","last_became_active":"2024-03-28T01:07:03.972014+0000","last_became_peered":"2024-03-28T01:07:03.972014+0000","last_unstale":"2024-03-28T01:08:24.183967+0000","last_undegraded":"2024-03-28T01:08:24.183967+0000","last_fullsized":"2024-03-28T01:08:24.183967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172851+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:08:24.172851+0000","last_peered":"2024-03-28T01:08:24.172851+0000","last_clean":"2024-03-28T01:08:24.172851+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:08:24.172851+0000","last_undegraded":"2024-03-28T01:08:24.172851+0000","last_fullsized":"2024-03-28T01:08:24.172851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183405+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:08:24.183405+0000","last_peered":"2024-03-28T01:08:24.183405+0000","last_clean":"2024-03-28T01:08:24.183405+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:08:24.183405+0000","last_undegraded":"2024-03-28T01:08:24.183405+0000","last_fullsized":"2024-03-28T01:08:24.183405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168319+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:08:24.168319+0000","last_peered":"2024-03-28T01:08:24.168319+0000","last_clean":"2024-03-28T01:08:24.168319+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:08:24.168319+0000","last_undegraded":"2024-03-28T01:08:24.168319+0000","last_fullsized":"2024-03-28T01:08:24.168319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'16","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180766+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:08:24.180766+0000","last_peered":"2024-03-28T01:08:24.180766+0000","last_clean":"2024-03-28T01:08:24.180766+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:08:24.180766+0000","last_undegraded":"2024-03-28T01:08:24.180766+0000","last_fullsized":"2024-03-28T01:08:24.180766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177548+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:08:24.177548+0000","last_peered":"2024-03-28T01:08:24.177548+0000","last_clean":"2024-03-28T01:08:24.177548+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:08:24.177548+0000","last_undegraded":"2024-03-28T01:08:24.177548+0000","last_fullsized":"2024-03-28T01:08:24.177548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'4","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182503+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:08:24.182503+0000","last_peered":"2024-03-28T01:08:24.182503+0000","last_clean":"2024-03-28T01:08:24.182503+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:08:24.182503+0000","last_undegraded":"2024-03-28T01:08:24.182503+0000","last_fullsized":"2024-03-28T01:08:24.182503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'7","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172868+0000","last_change":"2024-03-28T01:07:03.942155+0000","last_active":"2024-03-28T01:08:24.172868+0000","last_peered":"2024-03-28T01:08:24.172868+0000","last_clean":"2024-03-28T01:08:24.172868+0000","last_became_active":"2024-03-28T01:07:03.941729+0000","last_became_peered":"2024-03-28T01:07:03.941729+0000","last_unstale":"2024-03-28T01:08:24.172868+0000","last_undegraded":"2024-03-28T01:08:24.172868+0000","last_fullsized":"2024-03-28T01:08:24.172868+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:13:26.015917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'8","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.158440+0000","last_change":"2024-03-28T01:07:03.930604+0000","last_active":"2024-03-28T01:08:24.158440+0000","last_peered":"2024-03-28T01:08:24.158440+0000","last_clean":"2024-03-28T01:08:24.158440+0000","last_became_active":"2024-03-28T01:07:03.929334+0000","last_became_peered":"2024-03-28T01:07:03.929334+0000","last_unstale":"2024-03-28T01:08:24.158440+0000","last_undegraded":"2024-03-28T01:08:24.158440+0000","last_fullsized":"2024-03-28T01:08:24.158440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179921+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:08:24.179921+0000","last_peered":"2024-03-28T01:08:24.179921+0000","last_clean":"2024-03-28T01:08:24.179921+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:08:24.179921+0000","last_undegraded":"2024-03-28T01:08:24.179921+0000","last_fullsized":"2024-03-28T01:08:24.179921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180248+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:08:24.180248+0000","last_peered":"2024-03-28T01:08:24.180248+0000","last_clean":"2024-03-28T01:08:24.180248+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:08:24.180248+0000","last_undegraded":"2024-03-28T01:08:24.180248+0000","last_fullsized":"2024-03-28T01:08:24.180248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.152922+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:08:24.152922+0000","last_peered":"2024-03-28T01:08:24.152922+0000","last_clean":"2024-03-28T01:08:24.152922+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:08:24.152922+0000","last_undegraded":"2024-03-28T01:08:24.152922+0000","last_fullsized":"2024-03-28T01:08:24.152922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140953+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:08:19.140953+0000","last_peered":"2024-03-28T01:08:19.140953+0000","last_clean":"2024-03-28T01:08:19.140953+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:08:19.140953+0000","last_undegraded":"2024-03-28T01:08:19.140953+0000","last_fullsized":"2024-03-28T01:08:19.140953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181010+0000","last_change":"2024-03-28T01:06:22.717986+0000","last_active":"2024-03-28T01:08:24.181010+0000","last_peered":"2024-03-28T01:08:24.181010+0000","last_clean":"2024-03-28T01:08:24.181010+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:08:24.181010+0000","last_undegraded":"2024-03-28T01:08:24.181010+0000","last_fullsized":"2024-03-28T01:08:24.181010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145219+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:08:19.145219+0000","last_peered":"2024-03-28T01:08:19.145219+0000","last_clean":"2024-03-28T01:08:19.145219+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:08:19.145219+0000","last_undegraded":"2024-03-28T01:08:19.145219+0000","last_fullsized":"2024-03-28T01:08:19.145219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140916+0000","last_change":"2024-03-28T01:06:21.721278+0000","last_active":"2024-03-28T01:08:19.140916+0000","last_peered":"2024-03-28T01:08:19.140916+0000","last_clean":"2024-03-28T01:08:19.140916+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:08:19.140916+0000","last_undegraded":"2024-03-28T01:08:19.140916+0000","last_fullsized":"2024-03-28T01:08:19.140916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177391+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:08:24.177391+0000","last_peered":"2024-03-28T01:08:24.177391+0000","last_clean":"2024-03-28T01:08:24.177391+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:08:24.177391+0000","last_undegraded":"2024-03-28T01:08:24.177391+0000","last_fullsized":"2024-03-28T01:08:24.177391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580460+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:08:19.580460+0000","last_peered":"2024-03-28T01:08:19.580460+0000","last_clean":"2024-03-28T01:08:19.580460+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:08:19.580460+0000","last_undegraded":"2024-03-28T01:08:19.580460+0000","last_fullsized":"2024-03-28T01:08:19.580460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.178018+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:08:24.178018+0000","last_peered":"2024-03-28T01:08:24.178018+0000","last_clean":"2024-03-28T01:08:24.178018+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:08:24.178018+0000","last_undegraded":"2024-03-28T01:08:24.178018+0000","last_fullsized":"2024-03-28T01:08:24.178018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145869+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:08:19.145869+0000","last_peered":"2024-03-28T01:08:19.145869+0000","last_clean":"2024-03-28T01:08:19.145869+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:08:19.145869+0000","last_undegraded":"2024-03-28T01:08:19.145869+0000","last_fullsized":"2024-03-28T01:08:19.145869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184253+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:08:24.184253+0000","last_peered":"2024-03-28T01:08:24.184253+0000","last_clean":"2024-03-28T01:08:24.184253+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:08:24.184253+0000","last_undegraded":"2024-03-28T01:08:24.184253+0000","last_fullsized":"2024-03-28T01:08:24.184253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582176+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:08:19.582176+0000","last_peered":"2024-03-28T01:08:19.582176+0000","last_clean":"2024-03-28T01:08:19.582176+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:08:19.582176+0000","last_undegraded":"2024-03-28T01:08:19.582176+0000","last_fullsized":"2024-03-28T01:08:19.582176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'8","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.185188+0000","last_change":"2024-03-28T01:07:03.970890+0000","last_active":"2024-03-28T01:08:24.185188+0000","last_peered":"2024-03-28T01:08:24.185188+0000","last_clean":"2024-03-28T01:08:24.185188+0000","last_became_active":"2024-03-28T01:07:03.970312+0000","last_became_peered":"2024-03-28T01:07:03.970312+0000","last_unstale":"2024-03-28T01:08:24.185188+0000","last_undegraded":"2024-03-28T01:08:24.185188+0000","last_fullsized":"2024-03-28T01:08:24.185188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139870+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:08:19.139870+0000","last_peered":"2024-03-28T01:08:19.139870+0000","last_clean":"2024-03-28T01:08:19.139870+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:08:19.139870+0000","last_undegraded":"2024-03-28T01:08:19.139870+0000","last_fullsized":"2024-03-28T01:08:19.139870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179615+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:08:24.179615+0000","last_peered":"2024-03-28T01:08:24.179615+0000","last_clean":"2024-03-28T01:08:24.179615+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:08:24.179615+0000","last_undegraded":"2024-03-28T01:08:24.179615+0000","last_fullsized":"2024-03-28T01:08:24.179615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'111","reported_seq":131,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.200095+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:08:24.200095+0000","last_peered":"2024-03-28T01:08:24.200095+0000","last_clean":"2024-03-28T01:08:24.200095+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:08:24.200095+0000","last_undegraded":"2024-03-28T01:08:24.200095+0000","last_fullsized":"2024-03-28T01:08:24.200095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.193557+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:08:24.193557+0000","last_peered":"2024-03-28T01:08:24.193557+0000","last_clean":"2024-03-28T01:08:24.193557+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:08:24.193557+0000","last_undegraded":"2024-03-28T01:08:24.193557+0000","last_fullsized":"2024-03-28T01:08:24.193557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580187+0000","last_change":"2024-03-28T01:06:21.727305+0000","last_active":"2024-03-28T01:08:19.580187+0000","last_peered":"2024-03-28T01:08:19.580187+0000","last_clean":"2024-03-28T01:08:19.580187+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:08:19.580187+0000","last_undegraded":"2024-03-28T01:08:19.580187+0000","last_fullsized":"2024-03-28T01:08:19.580187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'11","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179008+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:08:24.179008+0000","last_peered":"2024-03-28T01:08:24.179008+0000","last_clean":"2024-03-28T01:08:24.179008+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:08:24.179008+0000","last_undegraded":"2024-03-28T01:08:24.179008+0000","last_fullsized":"2024-03-28T01:08:24.179008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580799+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:08:19.580799+0000","last_peered":"2024-03-28T01:08:19.580799+0000","last_clean":"2024-03-28T01:08:19.580799+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:08:19.580799+0000","last_undegraded":"2024-03-28T01:08:19.580799+0000","last_fullsized":"2024-03-28T01:08:19.580799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'15","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183597+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:08:24.183597+0000","last_peered":"2024-03-28T01:08:24.183597+0000","last_clean":"2024-03-28T01:08:24.183597+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:08:24.183597+0000","last_undegraded":"2024-03-28T01:08:24.183597+0000","last_fullsized":"2024-03-28T01:08:24.183597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145073+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:08:19.145073+0000","last_peered":"2024-03-28T01:08:19.145073+0000","last_clean":"2024-03-28T01:08:19.145073+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:08:19.145073+0000","last_undegraded":"2024-03-28T01:08:19.145073+0000","last_fullsized":"2024-03-28T01:08:19.145073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.158153+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:08:24.158153+0000","last_peered":"2024-03-28T01:08:24.158153+0000","last_clean":"2024-03-28T01:08:24.158153+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:08:24.158153+0000","last_undegraded":"2024-03-28T01:08:24.158153+0000","last_fullsized":"2024-03-28T01:08:24.158153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.579911+0000","last_change":"2024-03-28T01:06:21.719466+0000","last_active":"2024-03-28T01:08:19.579911+0000","last_peered":"2024-03-28T01:08:19.579911+0000","last_clean":"2024-03-28T01:08:19.579911+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:08:19.579911+0000","last_undegraded":"2024-03-28T01:08:19.579911+0000","last_fullsized":"2024-03-28T01:08:19.579911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181084+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:08:24.181084+0000","last_peered":"2024-03-28T01:08:24.181084+0000","last_clean":"2024-03-28T01:08:24.181084+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:08:24.181084+0000","last_undegraded":"2024-03-28T01:08:24.181084+0000","last_fullsized":"2024-03-28T01:08:24.181084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.142534+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:08:19.142534+0000","last_peered":"2024-03-28T01:08:19.142534+0000","last_clean":"2024-03-28T01:08:19.142534+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:08:19.142534+0000","last_undegraded":"2024-03-28T01:08:19.142534+0000","last_fullsized":"2024-03-28T01:08:19.142534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181248+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:08:24.181248+0000","last_peered":"2024-03-28T01:08:24.181248+0000","last_clean":"2024-03-28T01:08:24.181248+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:08:24.181248+0000","last_undegraded":"2024-03-28T01:08:24.181248+0000","last_fullsized":"2024-03-28T01:08:24.181248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580856+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:08:19.580856+0000","last_peered":"2024-03-28T01:08:19.580856+0000","last_clean":"2024-03-28T01:08:19.580856+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:08:19.580856+0000","last_undegraded":"2024-03-28T01:08:19.580856+0000","last_fullsized":"2024-03-28T01:08:19.580856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183144+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:08:24.183144+0000","last_peered":"2024-03-28T01:08:24.183144+0000","last_clean":"2024-03-28T01:08:24.183144+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:08:24.183144+0000","last_undegraded":"2024-03-28T01:08:24.183144+0000","last_fullsized":"2024-03-28T01:08:24.183144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139941+0000","last_change":"2024-03-28T01:06:21.718159+0000","last_active":"2024-03-28T01:08:19.139941+0000","last_peered":"2024-03-28T01:08:19.139941+0000","last_clean":"2024-03-28T01:08:19.139941+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:08:19.139941+0000","last_undegraded":"2024-03-28T01:08:19.139941+0000","last_fullsized":"2024-03-28T01:08:19.139941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182471+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:08:24.182471+0000","last_peered":"2024-03-28T01:08:24.182471+0000","last_clean":"2024-03-28T01:08:24.182471+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:08:24.182471+0000","last_undegraded":"2024-03-28T01:08:24.182471+0000","last_fullsized":"2024-03-28T01:08:24.182471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580700+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:08:19.580700+0000","last_peered":"2024-03-28T01:08:19.580700+0000","last_clean":"2024-03-28T01:08:19.580700+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:08:19.580700+0000","last_undegraded":"2024-03-28T01:08:19.580700+0000","last_fullsized":"2024-03-28T01:08:19.580700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182881+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:08:24.182881+0000","last_peered":"2024-03-28T01:08:24.182881+0000","last_clean":"2024-03-28T01:08:24.182881+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:08:24.182881+0000","last_undegraded":"2024-03-28T01:08:24.182881+0000","last_fullsized":"2024-03-28T01:08:24.182881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580583+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:08:19.580583+0000","last_peered":"2024-03-28T01:08:19.580583+0000","last_clean":"2024-03-28T01:08:19.580583+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:08:19.580583+0000","last_undegraded":"2024-03-28T01:08:19.580583+0000","last_fullsized":"2024-03-28T01:08:19.580583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580787+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:08:19.580787+0000","last_peered":"2024-03-28T01:08:19.580787+0000","last_clean":"2024-03-28T01:08:19.580787+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:08:19.580787+0000","last_undegraded":"2024-03-28T01:08:19.580787+0000","last_fullsized":"2024-03-28T01:08:19.580787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181907+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:08:24.181907+0000","last_peered":"2024-03-28T01:08:24.181907+0000","last_clean":"2024-03-28T01:08:24.181907+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:08:24.181907+0000","last_undegraded":"2024-03-28T01:08:24.181907+0000","last_fullsized":"2024-03-28T01:08:24.181907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580554+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:08:19.580554+0000","last_peered":"2024-03-28T01:08:19.580554+0000","last_clean":"2024-03-28T01:08:19.580554+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:08:19.580554+0000","last_undegraded":"2024-03-28T01:08:19.580554+0000","last_fullsized":"2024-03-28T01:08:19.580554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180800+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:08:24.180800+0000","last_peered":"2024-03-28T01:08:24.180800+0000","last_clean":"2024-03-28T01:08:24.180800+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:08:24.180800+0000","last_undegraded":"2024-03-28T01:08:24.180800+0000","last_fullsized":"2024-03-28T01:08:24.180800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'10","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.886078+0000","last_change":"2024-03-28T01:06:21.720753+0000","last_active":"2024-03-28T01:08:24.886078+0000","last_peered":"2024-03-28T01:08:24.886078+0000","last_clean":"2024-03-28T01:08:24.886078+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:08:24.886078+0000","last_undegraded":"2024-03-28T01:08:24.886078+0000","last_fullsized":"2024-03-28T01:08:24.886078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.161990+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:08:24.161990+0000","last_peered":"2024-03-28T01:08:24.161990+0000","last_clean":"2024-03-28T01:08:24.161990+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:08:24.161990+0000","last_undegraded":"2024-03-28T01:08:24.161990+0000","last_fullsized":"2024-03-28T01:08:24.161990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.581181+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:08:19.581181+0000","last_peered":"2024-03-28T01:08:19.581181+0000","last_clean":"2024-03-28T01:08:19.581181+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:08:19.581181+0000","last_undegraded":"2024-03-28T01:08:19.581181+0000","last_fullsized":"2024-03-28T01:08:19.581181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168298+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:08:24.168298+0000","last_peered":"2024-03-28T01:08:24.168298+0000","last_clean":"2024-03-28T01:08:24.168298+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:08:24.168298+0000","last_undegraded":"2024-03-28T01:08:24.168298+0000","last_fullsized":"2024-03-28T01:08:24.168298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139996+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:08:19.139996+0000","last_peered":"2024-03-28T01:08:19.139996+0000","last_clean":"2024-03-28T01:08:19.139996+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:08:19.139996+0000","last_undegraded":"2024-03-28T01:08:19.139996+0000","last_fullsized":"2024-03-28T01:08:19.139996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179379+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:08:24.179379+0000","last_peered":"2024-03-28T01:08:24.179379+0000","last_clean":"2024-03-28T01:08:24.179379+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:08:24.179379+0000","last_undegraded":"2024-03-28T01:08:24.179379+0000","last_fullsized":"2024-03-28T01:08:24.179379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582453+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:08:19.582453+0000","last_peered":"2024-03-28T01:08:19.582453+0000","last_clean":"2024-03-28T01:08:19.582453+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:08:19.582453+0000","last_undegraded":"2024-03-28T01:08:19.582453+0000","last_fullsized":"2024-03-28T01:08:19.582453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.166866+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:08:24.166866+0000","last_peered":"2024-03-28T01:08:24.166866+0000","last_clean":"2024-03-28T01:08:24.166866+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:08:24.166866+0000","last_undegraded":"2024-03-28T01:08:24.166866+0000","last_fullsized":"2024-03-28T01:08:24.166866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140107+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:08:19.140107+0000","last_peered":"2024-03-28T01:08:19.140107+0000","last_clean":"2024-03-28T01:08:19.140107+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:08:19.140107+0000","last_undegraded":"2024-03-28T01:08:19.140107+0000","last_fullsized":"2024-03-28T01:08:19.140107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:34.313143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183308+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:08:24.183308+0000","last_peered":"2024-03-28T01:08:24.183308+0000","last_clean":"2024-03-28T01:08:24.183308+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:08:24.183308+0000","last_undegraded":"2024-03-28T01:08:24.183308+0000","last_fullsized":"2024-03-28T01:08:24.183308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145990+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:08:19.145990+0000","last_peered":"2024-03-28T01:08:19.145990+0000","last_clean":"2024-03-28T01:08:19.145990+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:08:19.145990+0000","last_undegraded":"2024-03-28T01:08:19.145990+0000","last_fullsized":"2024-03-28T01:08:19.145990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171732+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:08:24.171732+0000","last_peered":"2024-03-28T01:08:24.171732+0000","last_clean":"2024-03-28T01:08:24.171732+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:08:24.171732+0000","last_undegraded":"2024-03-28T01:08:24.171732+0000","last_fullsized":"2024-03-28T01:08:24.171732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580100+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:08:19.580100+0000","last_peered":"2024-03-28T01:08:19.580100+0000","last_clean":"2024-03-28T01:08:19.580100+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:08:19.580100+0000","last_undegraded":"2024-03-28T01:08:19.580100+0000","last_fullsized":"2024-03-28T01:08:19.580100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140994+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:08:19.140994+0000","last_peered":"2024-03-28T01:08:19.140994+0000","last_clean":"2024-03-28T01:08:19.140994+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:08:19.140994+0000","last_undegraded":"2024-03-28T01:08:19.140994+0000","last_fullsized":"2024-03-28T01:08:19.140994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139443+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:08:19.139443+0000","last_peered":"2024-03-28T01:08:19.139443+0000","last_clean":"2024-03-28T01:08:19.139443+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:08:19.139443+0000","last_undegraded":"2024-03-28T01:08:19.139443+0000","last_fullsized":"2024-03-28T01:08:19.139443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168064+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:08:24.168064+0000","last_peered":"2024-03-28T01:08:24.168064+0000","last_clean":"2024-03-28T01:08:24.168064+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:08:24.168064+0000","last_undegraded":"2024-03-28T01:08:24.168064+0000","last_fullsized":"2024-03-28T01:08:24.168064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139655+0000","last_change":"2024-03-28T01:06:21.715183+0000","last_active":"2024-03-28T01:08:19.139655+0000","last_peered":"2024-03-28T01:08:19.139655+0000","last_clean":"2024-03-28T01:08:19.139655+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:08:19.139655+0000","last_undegraded":"2024-03-28T01:08:19.139655+0000","last_fullsized":"2024-03-28T01:08:19.139655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171388+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:08:24.171388+0000","last_peered":"2024-03-28T01:08:24.171388+0000","last_clean":"2024-03-28T01:08:24.171388+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:08:24.171388+0000","last_undegraded":"2024-03-28T01:08:24.171388+0000","last_fullsized":"2024-03-28T01:08:24.171388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582048+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:08:19.582048+0000","last_peered":"2024-03-28T01:08:19.582048+0000","last_clean":"2024-03-28T01:08:19.582048+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:08:19.582048+0000","last_undegraded":"2024-03-28T01:08:19.582048+0000","last_fullsized":"2024-03-28T01:08:19.582048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182659+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:08:24.182659+0000","last_peered":"2024-03-28T01:08:24.182659+0000","last_clean":"2024-03-28T01:08:24.182659+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:08:24.182659+0000","last_undegraded":"2024-03-28T01:08:24.182659+0000","last_fullsized":"2024-03-28T01:08:24.182659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'15","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182932+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:08:24.182932+0000","last_peered":"2024-03-28T01:08:24.182932+0000","last_clean":"2024-03-28T01:08:24.182932+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:08:24.182932+0000","last_undegraded":"2024-03-28T01:08:24.182932+0000","last_fullsized":"2024-03-28T01:08:24.182932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.581500+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:08:19.581500+0000","last_peered":"2024-03-28T01:08:19.581500+0000","last_clean":"2024-03-28T01:08:19.581500+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:08:19.581500+0000","last_undegraded":"2024-03-28T01:08:19.581500+0000","last_fullsized":"2024-03-28T01:08:19.581500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184715+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:08:24.184715+0000","last_peered":"2024-03-28T01:08:24.184715+0000","last_clean":"2024-03-28T01:08:24.184715+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:08:24.184715+0000","last_undegraded":"2024-03-28T01:08:24.184715+0000","last_fullsized":"2024-03-28T01:08:24.184715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582016+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:08:19.582016+0000","last_peered":"2024-03-28T01:08:19.582016+0000","last_clean":"2024-03-28T01:08:19.582016+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:08:19.582016+0000","last_undegraded":"2024-03-28T01:08:19.582016+0000","last_fullsized":"2024-03-28T01:08:19.582016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'6","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.173017+0000","last_change":"2024-03-28T01:07:03.942345+0000","last_active":"2024-03-28T01:08:24.173017+0000","last_peered":"2024-03-28T01:08:24.173017+0000","last_clean":"2024-03-28T01:08:24.173017+0000","last_became_active":"2024-03-28T01:07:03.941880+0000","last_became_peered":"2024-03-28T01:07:03.941880+0000","last_unstale":"2024-03-28T01:08:24.173017+0000","last_undegraded":"2024-03-28T01:08:24.173017+0000","last_fullsized":"2024-03-28T01:08:24.173017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:31:03.727315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.141335+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:08:19.141335+0000","last_peered":"2024-03-28T01:08:19.141335+0000","last_clean":"2024-03-28T01:08:19.141335+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:08:19.141335+0000","last_undegraded":"2024-03-28T01:08:19.141335+0000","last_fullsized":"2024-03-28T01:08:19.141335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:32.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177380+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:08:24.177380+0000","last_peered":"2024-03-28T01:08:24.177380+0000","last_clean":"2024-03-28T01:08:24.177380+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:08:24.177380+0000","last_undegraded":"2024-03-28T01:08:24.177380+0000","last_fullsized":"2024-03-28T01:08:24.177380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.579757+0000","last_change":"2024-03-28T01:06:21.727858+0000","last_active":"2024-03-28T01:08:19.579757+0000","last_peered":"2024-03-28T01:08:19.579757+0000","last_clean":"2024-03-28T01:08:19.579757+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:08:19.579757+0000","last_undegraded":"2024-03-28T01:08:19.579757+0000","last_fullsized":"2024-03-28T01:08:19.579757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:15:55.049218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":18,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":36,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":526,"ondisk_log_size":526,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":130821,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":170,"num_write_kb":213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":261642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":860160,"data_stored":680432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":218379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":89563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":89596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":397312,"data_stored":219499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":90180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":89627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":851968,"data_stored":680476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":130023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":128912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":9,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":6,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:08:29.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:08:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.410+0000 7f2362c00640 1 Processor -- start 2024-03-28T01:08:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.410+0000 7f2362c00640 1 -- start start 2024-03-28T01:08:29.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.411+0000 7f2362c00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f235c0b51d0 0x7f235c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2362c00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f235c0b8d00 0x7f235c14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2362c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c0b42f0 0x7f235c14b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2362c00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c14a820 con 0x7f235c0b51d0 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2362c00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c0b55b0 con 0x7f235c0b8d00 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2362c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b9190 con 0x7f235c0b42f0 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2361800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f235c0b51d0 0x7f235c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2361800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f235c0b51d0 0x7f235c0b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32792/0 (socket says 172.21.15.33:32792) 2024-03-28T01:08:29.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2361800640 1 -- 172.21.15.33:0/2283199172 learned_addr learned my addr 172.21.15.33:0/2283199172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2360e00640 1 --2- 172.21.15.33:0/2283199172 >> v2:172.21.15.194:3300/0 conn(0x7f235c0b8d00 0x7f235c14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.412+0000 7f2362200640 1 --2- 172.21.15.33:0/2283199172 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c0b42f0 0x7f235c14b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.413+0000 7f2361800640 1 -- 172.21.15.33:0/2283199172 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c0b42f0 msgr2=0x7f235c14b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.413+0000 7f2361800640 1 --2- 172.21.15.33:0/2283199172 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c0b42f0 0x7f235c14b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.413+0000 7f2361800640 1 -- 172.21.15.33:0/2283199172 >> v2:172.21.15.194:3300/0 conn(0x7f235c0b8d00 msgr2=0x7f235c14ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.413+0000 7f2361800640 1 --2- 172.21.15.33:0/2283199172 >> v2:172.21.15.194:3300/0 conn(0x7f235c0b8d00 0x7f235c14ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.413+0000 7f2361800640 1 -- 172.21.15.33:0/2283199172 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0b9800 con 0x7f235c0b51d0 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.414+0000 7f2361800640 1 --2- 172.21.15.33:0/2283199172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b51d0 0x7f235c0b3db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b4890 tx=0x7f235002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290986ac3cab9f1c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.415+0000 7f235be00640 1 -- 172.21.15.33:0/2283199172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350035020 con 0x7f235c0b51d0 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.415+0000 7f235be00640 1 -- 172.21.15.33:0/2283199172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2350031870 con 0x7f235c0b51d0 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.415+0000 7f235be00640 1 -- 172.21.15.33:0/2283199172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350031c40 con 0x7f235c0b51d0 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.415+0000 7f2362c00640 1 -- 172.21.15.33:0/2283199172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b51d0 msgr2=0x7f235c0b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.415+0000 7f2362c00640 1 --2- 172.21.15.33:0/2283199172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b51d0 0x7f235c0b3db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b4890 tx=0x7f235002b910 comp rx=0 tx=0).stop 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.416+0000 7f2362c00640 1 -- 172.21.15.33:0/2283199172 shutdown_connections 2024-03-28T01:08:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.416+0000 7f2362c00640 1 --2- 172.21.15.33:0/2283199172 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c0b42f0 0x7f235c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.416+0000 7f2362c00640 1 --2- 172.21.15.33:0/2283199172 >> v2:172.21.15.194:3300/0 conn(0x7f235c0b8d00 0x7f235c14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.416+0000 7f2362c00640 1 --2- 172.21.15.33:0/2283199172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b51d0 0x7f235c0b3db0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.416+0000 7f2362c00640 1 -- 172.21.15.33:0/2283199172 >> 172.21.15.33:0/2283199172 conn(0x7f235c072940 msgr2=0x7f235c072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.416+0000 7f2362c00640 1 -- 172.21.15.33:0/2283199172 shutdown_connections 2024-03-28T01:08:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.416+0000 7f2362c00640 1 -- 172.21.15.33:0/2283199172 wait complete. 2024-03-28T01:08:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.418+0000 7f2362c00640 1 Processor -- start 2024-03-28T01:08:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.419+0000 7f2362c00640 1 -- start start 2024-03-28T01:08:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2362c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b42f0 0x7f235c0bc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2362c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235c0b51d0 0x7f235c142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2362c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c144490 0x7f235c142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2362c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b39c0 con 0x7f235c0b42f0 2024-03-28T01:08:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2362c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c14b6f0 con 0x7f235c0b51d0 2024-03-28T01:08:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2362c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b6b80 con 0x7f235c144490 2024-03-28T01:08:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2361800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b42f0 0x7f235c0bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2361800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b42f0 0x7f235c0bc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32798/0 (socket says 172.21.15.33:32798) 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.420+0000 7f2361800640 1 -- 172.21.15.33:0/989691797 learned_addr learned my addr 172.21.15.33:0/989691797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2362200640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c144490 0x7f235c142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2360e00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235c0b51d0 0x7f235c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2361800640 1 -- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c144490 msgr2=0x7f235c142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2361800640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c144490 0x7f235c142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2361800640 1 -- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235c0b51d0 msgr2=0x7f235c142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2361800640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235c0b51d0 0x7f235c142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2361800640 1 -- 172.21.15.33:0/989691797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c1d1030 con 0x7f235c0b42f0 2024-03-28T01:08:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2361800640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b42f0 0x7f235c0bc760 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2350000c00 tx=0x7f23500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.421+0000 7f2359600640 1 -- 172.21.15.33:0/989691797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350035040 con 0x7f235c0b42f0 2024-03-28T01:08:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.422+0000 7f2359600640 1 -- 172.21.15.33:0/989691797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2350030d00 con 0x7f235c0b42f0 2024-03-28T01:08:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.422+0000 7f2359600640 1 -- 172.21.15.33:0/989691797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23500456a0 con 0x7f235c0b42f0 2024-03-28T01:08:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.422+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c145050 con 0x7f235c0b42f0 2024-03-28T01:08:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.422+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c144d70 con 0x7f235c0b42f0 2024-03-28T01:08:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.423+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c075310 con 0x7f235c0b42f0 2024-03-28T01:08:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.424+0000 7f2359600640 1 -- 172.21.15.33:0/989691797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2350038020 con 0x7f235c0b42f0 2024-03-28T01:08:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.425+0000 7f2359600640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f233c06bc00 0x7f233c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.425+0000 7f2359600640 1 -- 172.21.15.33:0/989691797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f23500ace80 con 0x7f235c0b42f0 2024-03-28T01:08:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.426+0000 7f2360e00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f233c06bc00 0x7f233c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.426+0000 7f2360e00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f233c06bc00 0x7f233c06e0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2348006fd0 tx=0x7f2348008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.430+0000 7f2359600640 1 -- 172.21.15.33:0/989691797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f235007b880 con 0x7f235c0b42f0 2024-03-28T01:08:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.664+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f235c146d20 con 0x7f233c06bc00 2024-03-28T01:08:29.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.672+0000 7f2359600640 1 -- 172.21.15.33:0/989691797 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252146 (secure 0 0 0) 0x7f235c146d20 con 0x7f233c06bc00 2024-03-28T01:08:29.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:08:29.679 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:08:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f233c06bc00 msgr2=0x7f233c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f233c06bc00 0x7f233c06e0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2348006fd0 tx=0x7f2348008040 comp rx=0 tx=0).stop 2024-03-28T01:08:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b42f0 msgr2=0x7f235c0bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b42f0 0x7f235c0bc760 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2350000c00 tx=0x7f23500046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 shutdown_connections 2024-03-28T01:08:29.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f233c06bc00 0x7f233c06e0c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f235c144490 0x7f235c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235c0b51d0 0x7f235c142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 --2- 172.21.15.33:0/989691797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235c0b42f0 0x7f235c0bc760 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 >> 172.21.15.33:0/989691797 conn(0x7f235c072940 msgr2=0x7f235c146380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:29.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.678+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 shutdown_connections 2024-03-28T01:08:29.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.679+0000 7f2362c00640 1 -- 172.21.15.33:0/989691797 wait complete. 2024-03-28T01:08:29.699 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-03-28T01:08:28.428123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":721207,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":125,"num_read_kb":109,"num_write":345,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":813,"ondisk_log_size":813,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219552,"kb_used_data":3424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755168,"statfs":{"total":773094113280,"available":772869292032,"internally_reserved":0,"allocated":3506176,"data_stored":2157752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117370,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003984"},"pg_stats":[{"pgid":"3.3f","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182684+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:08:24.182684+0000","last_peered":"2024-03-28T01:08:24.182684+0000","last_clean":"2024-03-28T01:08:24.182684+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:08:24.182684+0000","last_undegraded":"2024-03-28T01:08:24.182684+0000","last_fullsized":"2024-03-28T01:08:24.182684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'4","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.170596+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:08:24.170596+0000","last_peered":"2024-03-28T01:08:24.170596+0000","last_clean":"2024-03-28T01:08:24.170596+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:08:24.170596+0000","last_undegraded":"2024-03-28T01:08:24.170596+0000","last_fullsized":"2024-03-28T01:08:24.170596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.170497+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:08:24.170497+0000","last_peered":"2024-03-28T01:08:24.170497+0000","last_clean":"2024-03-28T01:08:24.170497+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:08:24.170497+0000","last_undegraded":"2024-03-28T01:08:24.170497+0000","last_fullsized":"2024-03-28T01:08:24.170497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184333+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:08:24.184333+0000","last_peered":"2024-03-28T01:08:24.184333+0000","last_clean":"2024-03-28T01:08:24.184333+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:08:24.184333+0000","last_undegraded":"2024-03-28T01:08:24.184333+0000","last_fullsized":"2024-03-28T01:08:24.184333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183192+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:08:24.183192+0000","last_peered":"2024-03-28T01:08:24.183192+0000","last_clean":"2024-03-28T01:08:24.183192+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:08:24.183192+0000","last_undegraded":"2024-03-28T01:08:24.183192+0000","last_fullsized":"2024-03-28T01:08:24.183192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172793+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:08:24.172793+0000","last_peered":"2024-03-28T01:08:24.172793+0000","last_clean":"2024-03-28T01:08:24.172793+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:08:24.172793+0000","last_undegraded":"2024-03-28T01:08:24.172793+0000","last_fullsized":"2024-03-28T01:08:24.172793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.166510+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:08:24.166510+0000","last_peered":"2024-03-28T01:08:24.166510+0000","last_clean":"2024-03-28T01:08:24.166510+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:08:24.166510+0000","last_undegraded":"2024-03-28T01:08:24.166510+0000","last_fullsized":"2024-03-28T01:08:24.166510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183817+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:08:24.183817+0000","last_peered":"2024-03-28T01:08:24.183817+0000","last_clean":"2024-03-28T01:08:24.183817+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:08:24.183817+0000","last_undegraded":"2024-03-28T01:08:24.183817+0000","last_fullsized":"2024-03-28T01:08:24.183817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.185365+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:08:24.185365+0000","last_peered":"2024-03-28T01:08:24.185365+0000","last_clean":"2024-03-28T01:08:24.185365+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:08:24.185365+0000","last_undegraded":"2024-03-28T01:08:24.185365+0000","last_fullsized":"2024-03-28T01:08:24.185365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'17","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184340+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:08:24.184340+0000","last_peered":"2024-03-28T01:08:24.184340+0000","last_clean":"2024-03-28T01:08:24.184340+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:08:24.184340+0000","last_undegraded":"2024-03-28T01:08:24.184340+0000","last_fullsized":"2024-03-28T01:08:24.184340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183682+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:08:24.183682+0000","last_peered":"2024-03-28T01:08:24.183682+0000","last_clean":"2024-03-28T01:08:24.183682+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:08:24.183682+0000","last_undegraded":"2024-03-28T01:08:24.183682+0000","last_fullsized":"2024-03-28T01:08:24.183682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.161730+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:08:24.161730+0000","last_peered":"2024-03-28T01:08:24.161730+0000","last_clean":"2024-03-28T01:08:24.161730+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:08:24.161730+0000","last_undegraded":"2024-03-28T01:08:24.161730+0000","last_fullsized":"2024-03-28T01:08:24.161730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182627+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:08:24.182627+0000","last_peered":"2024-03-28T01:08:24.182627+0000","last_clean":"2024-03-28T01:08:24.182627+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:08:24.182627+0000","last_undegraded":"2024-03-28T01:08:24.182627+0000","last_fullsized":"2024-03-28T01:08:24.182627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172340+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:08:24.172340+0000","last_peered":"2024-03-28T01:08:24.172340+0000","last_clean":"2024-03-28T01:08:24.172340+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:08:24.172340+0000","last_undegraded":"2024-03-28T01:08:24.172340+0000","last_fullsized":"2024-03-28T01:08:24.172340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171787+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:08:24.171787+0000","last_peered":"2024-03-28T01:08:24.171787+0000","last_clean":"2024-03-28T01:08:24.171787+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:08:24.171787+0000","last_undegraded":"2024-03-28T01:08:24.171787+0000","last_fullsized":"2024-03-28T01:08:24.171787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.173992+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:08:24.173992+0000","last_peered":"2024-03-28T01:08:24.173992+0000","last_clean":"2024-03-28T01:08:24.173992+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:08:24.173992+0000","last_undegraded":"2024-03-28T01:08:24.173992+0000","last_fullsized":"2024-03-28T01:08:24.173992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.176879+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:08:24.176879+0000","last_peered":"2024-03-28T01:08:24.176879+0000","last_clean":"2024-03-28T01:08:24.176879+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:08:24.176879+0000","last_undegraded":"2024-03-28T01:08:24.176879+0000","last_fullsized":"2024-03-28T01:08:24.176879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179164+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:08:24.179164+0000","last_peered":"2024-03-28T01:08:24.179164+0000","last_clean":"2024-03-28T01:08:24.179164+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:08:24.179164+0000","last_undegraded":"2024-03-28T01:08:24.179164+0000","last_fullsized":"2024-03-28T01:08:24.179164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'7","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.169769+0000","last_change":"2024-03-28T01:07:03.939021+0000","last_active":"2024-03-28T01:08:24.169769+0000","last_peered":"2024-03-28T01:08:24.169769+0000","last_clean":"2024-03-28T01:08:24.169769+0000","last_became_active":"2024-03-28T01:07:03.937565+0000","last_became_peered":"2024-03-28T01:07:03.937565+0000","last_unstale":"2024-03-28T01:08:24.169769+0000","last_undegraded":"2024-03-28T01:08:24.169769+0000","last_fullsized":"2024-03-28T01:08:24.169769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:13:26.015917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181877+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:08:24.181877+0000","last_peered":"2024-03-28T01:08:24.181877+0000","last_clean":"2024-03-28T01:08:24.181877+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:08:24.181877+0000","last_undegraded":"2024-03-28T01:08:24.181877+0000","last_fullsized":"2024-03-28T01:08:24.181877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.178531+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:08:24.178531+0000","last_peered":"2024-03-28T01:08:24.178531+0000","last_clean":"2024-03-28T01:08:24.178531+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:08:24.178531+0000","last_undegraded":"2024-03-28T01:08:24.178531+0000","last_fullsized":"2024-03-28T01:08:24.178531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183967+0000","last_change":"2024-03-28T01:07:03.972650+0000","last_active":"2024-03-28T01:08:24.183967+0000","last_peered":"2024-03-28T01:08:24.183967+0000","last_clean":"2024-03-28T01:08:24.183967+0000","last_became_active":"2024-03-28T01:07:03.972014+0000","last_became_peered":"2024-03-28T01:07:03.972014+0000","last_unstale":"2024-03-28T01:08:24.183967+0000","last_undegraded":"2024-03-28T01:08:24.183967+0000","last_fullsized":"2024-03-28T01:08:24.183967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172851+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:08:24.172851+0000","last_peered":"2024-03-28T01:08:24.172851+0000","last_clean":"2024-03-28T01:08:24.172851+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:08:24.172851+0000","last_undegraded":"2024-03-28T01:08:24.172851+0000","last_fullsized":"2024-03-28T01:08:24.172851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183405+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:08:24.183405+0000","last_peered":"2024-03-28T01:08:24.183405+0000","last_clean":"2024-03-28T01:08:24.183405+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:08:24.183405+0000","last_undegraded":"2024-03-28T01:08:24.183405+0000","last_fullsized":"2024-03-28T01:08:24.183405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168319+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:08:24.168319+0000","last_peered":"2024-03-28T01:08:24.168319+0000","last_clean":"2024-03-28T01:08:24.168319+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:08:24.168319+0000","last_undegraded":"2024-03-28T01:08:24.168319+0000","last_fullsized":"2024-03-28T01:08:24.168319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'16","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180766+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:08:24.180766+0000","last_peered":"2024-03-28T01:08:24.180766+0000","last_clean":"2024-03-28T01:08:24.180766+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:08:24.180766+0000","last_undegraded":"2024-03-28T01:08:24.180766+0000","last_fullsized":"2024-03-28T01:08:24.180766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177548+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:08:24.177548+0000","last_peered":"2024-03-28T01:08:24.177548+0000","last_clean":"2024-03-28T01:08:24.177548+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:08:24.177548+0000","last_undegraded":"2024-03-28T01:08:24.177548+0000","last_fullsized":"2024-03-28T01:08:24.177548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'4","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182503+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:08:24.182503+0000","last_peered":"2024-03-28T01:08:24.182503+0000","last_clean":"2024-03-28T01:08:24.182503+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:08:24.182503+0000","last_undegraded":"2024-03-28T01:08:24.182503+0000","last_fullsized":"2024-03-28T01:08:24.182503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'7","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172868+0000","last_change":"2024-03-28T01:07:03.942155+0000","last_active":"2024-03-28T01:08:24.172868+0000","last_peered":"2024-03-28T01:08:24.172868+0000","last_clean":"2024-03-28T01:08:24.172868+0000","last_became_active":"2024-03-28T01:07:03.941729+0000","last_became_peered":"2024-03-28T01:07:03.941729+0000","last_unstale":"2024-03-28T01:08:24.172868+0000","last_undegraded":"2024-03-28T01:08:24.172868+0000","last_fullsized":"2024-03-28T01:08:24.172868+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:13:26.015917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'8","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.158440+0000","last_change":"2024-03-28T01:07:03.930604+0000","last_active":"2024-03-28T01:08:24.158440+0000","last_peered":"2024-03-28T01:08:24.158440+0000","last_clean":"2024-03-28T01:08:24.158440+0000","last_became_active":"2024-03-28T01:07:03.929334+0000","last_became_peered":"2024-03-28T01:07:03.929334+0000","last_unstale":"2024-03-28T01:08:24.158440+0000","last_undegraded":"2024-03-28T01:08:24.158440+0000","last_fullsized":"2024-03-28T01:08:24.158440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179921+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:08:24.179921+0000","last_peered":"2024-03-28T01:08:24.179921+0000","last_clean":"2024-03-28T01:08:24.179921+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:08:24.179921+0000","last_undegraded":"2024-03-28T01:08:24.179921+0000","last_fullsized":"2024-03-28T01:08:24.179921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180248+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:08:24.180248+0000","last_peered":"2024-03-28T01:08:24.180248+0000","last_clean":"2024-03-28T01:08:24.180248+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:08:24.180248+0000","last_undegraded":"2024-03-28T01:08:24.180248+0000","last_fullsized":"2024-03-28T01:08:24.180248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.152922+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:08:24.152922+0000","last_peered":"2024-03-28T01:08:24.152922+0000","last_clean":"2024-03-28T01:08:24.152922+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:08:24.152922+0000","last_undegraded":"2024-03-28T01:08:24.152922+0000","last_fullsized":"2024-03-28T01:08:24.152922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140953+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:08:19.140953+0000","last_peered":"2024-03-28T01:08:19.140953+0000","last_clean":"2024-03-28T01:08:19.140953+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:08:19.140953+0000","last_undegraded":"2024-03-28T01:08:19.140953+0000","last_fullsized":"2024-03-28T01:08:19.140953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181010+0000","last_change":"2024-03-28T01:06:22.717986+0000","last_active":"2024-03-28T01:08:24.181010+0000","last_peered":"2024-03-28T01:08:24.181010+0000","last_clean":"2024-03-28T01:08:24.181010+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:08:24.181010+0000","last_undegraded":"2024-03-28T01:08:24.181010+0000","last_fullsized":"2024-03-28T01:08:24.181010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145219+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:08:19.145219+0000","last_peered":"2024-03-28T01:08:19.145219+0000","last_clean":"2024-03-28T01:08:19.145219+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:08:19.145219+0000","last_undegraded":"2024-03-28T01:08:19.145219+0000","last_fullsized":"2024-03-28T01:08:19.145219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140916+0000","last_change":"2024-03-28T01:06:21.721278+0000","last_active":"2024-03-28T01:08:19.140916+0000","last_peered":"2024-03-28T01:08:19.140916+0000","last_clean":"2024-03-28T01:08:19.140916+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:08:19.140916+0000","last_undegraded":"2024-03-28T01:08:19.140916+0000","last_fullsized":"2024-03-28T01:08:19.140916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177391+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:08:24.177391+0000","last_peered":"2024-03-28T01:08:24.177391+0000","last_clean":"2024-03-28T01:08:24.177391+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:08:24.177391+0000","last_undegraded":"2024-03-28T01:08:24.177391+0000","last_fullsized":"2024-03-28T01:08:24.177391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580460+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:08:19.580460+0000","last_peered":"2024-03-28T01:08:19.580460+0000","last_clean":"2024-03-28T01:08:19.580460+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:08:19.580460+0000","last_undegraded":"2024-03-28T01:08:19.580460+0000","last_fullsized":"2024-03-28T01:08:19.580460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.178018+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:08:24.178018+0000","last_peered":"2024-03-28T01:08:24.178018+0000","last_clean":"2024-03-28T01:08:24.178018+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:08:24.178018+0000","last_undegraded":"2024-03-28T01:08:24.178018+0000","last_fullsized":"2024-03-28T01:08:24.178018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145869+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:08:19.145869+0000","last_peered":"2024-03-28T01:08:19.145869+0000","last_clean":"2024-03-28T01:08:19.145869+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:08:19.145869+0000","last_undegraded":"2024-03-28T01:08:19.145869+0000","last_fullsized":"2024-03-28T01:08:19.145869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184253+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:08:24.184253+0000","last_peered":"2024-03-28T01:08:24.184253+0000","last_clean":"2024-03-28T01:08:24.184253+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:08:24.184253+0000","last_undegraded":"2024-03-28T01:08:24.184253+0000","last_fullsized":"2024-03-28T01:08:24.184253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582176+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:08:19.582176+0000","last_peered":"2024-03-28T01:08:19.582176+0000","last_clean":"2024-03-28T01:08:19.582176+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:08:19.582176+0000","last_undegraded":"2024-03-28T01:08:19.582176+0000","last_fullsized":"2024-03-28T01:08:19.582176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'8","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.185188+0000","last_change":"2024-03-28T01:07:03.970890+0000","last_active":"2024-03-28T01:08:24.185188+0000","last_peered":"2024-03-28T01:08:24.185188+0000","last_clean":"2024-03-28T01:08:24.185188+0000","last_became_active":"2024-03-28T01:07:03.970312+0000","last_became_peered":"2024-03-28T01:07:03.970312+0000","last_unstale":"2024-03-28T01:08:24.185188+0000","last_undegraded":"2024-03-28T01:08:24.185188+0000","last_fullsized":"2024-03-28T01:08:24.185188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139870+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:08:19.139870+0000","last_peered":"2024-03-28T01:08:19.139870+0000","last_clean":"2024-03-28T01:08:19.139870+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:08:19.139870+0000","last_undegraded":"2024-03-28T01:08:19.139870+0000","last_fullsized":"2024-03-28T01:08:19.139870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179615+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:08:24.179615+0000","last_peered":"2024-03-28T01:08:24.179615+0000","last_clean":"2024-03-28T01:08:24.179615+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:08:24.179615+0000","last_undegraded":"2024-03-28T01:08:24.179615+0000","last_fullsized":"2024-03-28T01:08:24.179615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'111","reported_seq":131,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.200095+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:08:24.200095+0000","last_peered":"2024-03-28T01:08:24.200095+0000","last_clean":"2024-03-28T01:08:24.200095+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:08:24.200095+0000","last_undegraded":"2024-03-28T01:08:24.200095+0000","last_fullsized":"2024-03-28T01:08:24.200095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.193557+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:08:24.193557+0000","last_peered":"2024-03-28T01:08:24.193557+0000","last_clean":"2024-03-28T01:08:24.193557+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:08:24.193557+0000","last_undegraded":"2024-03-28T01:08:24.193557+0000","last_fullsized":"2024-03-28T01:08:24.193557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580187+0000","last_change":"2024-03-28T01:06:21.727305+0000","last_active":"2024-03-28T01:08:19.580187+0000","last_peered":"2024-03-28T01:08:19.580187+0000","last_clean":"2024-03-28T01:08:19.580187+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:08:19.580187+0000","last_undegraded":"2024-03-28T01:08:19.580187+0000","last_fullsized":"2024-03-28T01:08:19.580187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'11","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179008+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:08:24.179008+0000","last_peered":"2024-03-28T01:08:24.179008+0000","last_clean":"2024-03-28T01:08:24.179008+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:08:24.179008+0000","last_undegraded":"2024-03-28T01:08:24.179008+0000","last_fullsized":"2024-03-28T01:08:24.179008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580799+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:08:19.580799+0000","last_peered":"2024-03-28T01:08:19.580799+0000","last_clean":"2024-03-28T01:08:19.580799+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:08:19.580799+0000","last_undegraded":"2024-03-28T01:08:19.580799+0000","last_fullsized":"2024-03-28T01:08:19.580799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'15","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183597+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:08:24.183597+0000","last_peered":"2024-03-28T01:08:24.183597+0000","last_clean":"2024-03-28T01:08:24.183597+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:08:24.183597+0000","last_undegraded":"2024-03-28T01:08:24.183597+0000","last_fullsized":"2024-03-28T01:08:24.183597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145073+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:08:19.145073+0000","last_peered":"2024-03-28T01:08:19.145073+0000","last_clean":"2024-03-28T01:08:19.145073+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:08:19.145073+0000","last_undegraded":"2024-03-28T01:08:19.145073+0000","last_fullsized":"2024-03-28T01:08:19.145073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.158153+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:08:24.158153+0000","last_peered":"2024-03-28T01:08:24.158153+0000","last_clean":"2024-03-28T01:08:24.158153+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:08:24.158153+0000","last_undegraded":"2024-03-28T01:08:24.158153+0000","last_fullsized":"2024-03-28T01:08:24.158153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.579911+0000","last_change":"2024-03-28T01:06:21.719466+0000","last_active":"2024-03-28T01:08:19.579911+0000","last_peered":"2024-03-28T01:08:19.579911+0000","last_clean":"2024-03-28T01:08:19.579911+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:08:19.579911+0000","last_undegraded":"2024-03-28T01:08:19.579911+0000","last_fullsized":"2024-03-28T01:08:19.579911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181084+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:08:24.181084+0000","last_peered":"2024-03-28T01:08:24.181084+0000","last_clean":"2024-03-28T01:08:24.181084+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:08:24.181084+0000","last_undegraded":"2024-03-28T01:08:24.181084+0000","last_fullsized":"2024-03-28T01:08:24.181084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.142534+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:08:19.142534+0000","last_peered":"2024-03-28T01:08:19.142534+0000","last_clean":"2024-03-28T01:08:19.142534+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:08:19.142534+0000","last_undegraded":"2024-03-28T01:08:19.142534+0000","last_fullsized":"2024-03-28T01:08:19.142534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181248+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:08:24.181248+0000","last_peered":"2024-03-28T01:08:24.181248+0000","last_clean":"2024-03-28T01:08:24.181248+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:08:24.181248+0000","last_undegraded":"2024-03-28T01:08:24.181248+0000","last_fullsized":"2024-03-28T01:08:24.181248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580856+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:08:19.580856+0000","last_peered":"2024-03-28T01:08:19.580856+0000","last_clean":"2024-03-28T01:08:19.580856+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:08:19.580856+0000","last_undegraded":"2024-03-28T01:08:19.580856+0000","last_fullsized":"2024-03-28T01:08:19.580856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183144+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:08:24.183144+0000","last_peered":"2024-03-28T01:08:24.183144+0000","last_clean":"2024-03-28T01:08:24.183144+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:08:24.183144+0000","last_undegraded":"2024-03-28T01:08:24.183144+0000","last_fullsized":"2024-03-28T01:08:24.183144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139941+0000","last_change":"2024-03-28T01:06:21.718159+0000","last_active":"2024-03-28T01:08:19.139941+0000","last_peered":"2024-03-28T01:08:19.139941+0000","last_clean":"2024-03-28T01:08:19.139941+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:08:19.139941+0000","last_undegraded":"2024-03-28T01:08:19.139941+0000","last_fullsized":"2024-03-28T01:08:19.139941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182471+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:08:24.182471+0000","last_peered":"2024-03-28T01:08:24.182471+0000","last_clean":"2024-03-28T01:08:24.182471+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:08:24.182471+0000","last_undegraded":"2024-03-28T01:08:24.182471+0000","last_fullsized":"2024-03-28T01:08:24.182471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580700+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:08:19.580700+0000","last_peered":"2024-03-28T01:08:19.580700+0000","last_clean":"2024-03-28T01:08:19.580700+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:08:19.580700+0000","last_undegraded":"2024-03-28T01:08:19.580700+0000","last_fullsized":"2024-03-28T01:08:19.580700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182881+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:08:24.182881+0000","last_peered":"2024-03-28T01:08:24.182881+0000","last_clean":"2024-03-28T01:08:24.182881+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:08:24.182881+0000","last_undegraded":"2024-03-28T01:08:24.182881+0000","last_fullsized":"2024-03-28T01:08:24.182881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580583+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:08:19.580583+0000","last_peered":"2024-03-28T01:08:19.580583+0000","last_clean":"2024-03-28T01:08:19.580583+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:08:19.580583+0000","last_undegraded":"2024-03-28T01:08:19.580583+0000","last_fullsized":"2024-03-28T01:08:19.580583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580787+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:08:19.580787+0000","last_peered":"2024-03-28T01:08:19.580787+0000","last_clean":"2024-03-28T01:08:19.580787+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:08:19.580787+0000","last_undegraded":"2024-03-28T01:08:19.580787+0000","last_fullsized":"2024-03-28T01:08:19.580787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181907+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:08:24.181907+0000","last_peered":"2024-03-28T01:08:24.181907+0000","last_clean":"2024-03-28T01:08:24.181907+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:08:24.181907+0000","last_undegraded":"2024-03-28T01:08:24.181907+0000","last_fullsized":"2024-03-28T01:08:24.181907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580554+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:08:19.580554+0000","last_peered":"2024-03-28T01:08:19.580554+0000","last_clean":"2024-03-28T01:08:19.580554+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:08:19.580554+0000","last_undegraded":"2024-03-28T01:08:19.580554+0000","last_fullsized":"2024-03-28T01:08:19.580554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180800+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:08:24.180800+0000","last_peered":"2024-03-28T01:08:24.180800+0000","last_clean":"2024-03-28T01:08:24.180800+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:08:24.180800+0000","last_undegraded":"2024-03-28T01:08:24.180800+0000","last_fullsized":"2024-03-28T01:08:24.180800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'10","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.886078+0000","last_change":"2024-03-28T01:06:21.720753+0000","last_active":"2024-03-28T01:08:24.886078+0000","last_peered":"2024-03-28T01:08:24.886078+0000","last_clean":"2024-03-28T01:08:24.886078+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:08:24.886078+0000","last_undegraded":"2024-03-28T01:08:24.886078+0000","last_fullsized":"2024-03-28T01:08:24.886078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.161990+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:08:24.161990+0000","last_peered":"2024-03-28T01:08:24.161990+0000","last_clean":"2024-03-28T01:08:24.161990+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:08:24.161990+0000","last_undegraded":"2024-03-28T01:08:24.161990+0000","last_fullsized":"2024-03-28T01:08:24.161990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.581181+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:08:19.581181+0000","last_peered":"2024-03-28T01:08:19.581181+0000","last_clean":"2024-03-28T01:08:19.581181+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:08:19.581181+0000","last_undegraded":"2024-03-28T01:08:19.581181+0000","last_fullsized":"2024-03-28T01:08:19.581181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168298+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:08:24.168298+0000","last_peered":"2024-03-28T01:08:24.168298+0000","last_clean":"2024-03-28T01:08:24.168298+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:08:24.168298+0000","last_undegraded":"2024-03-28T01:08:24.168298+0000","last_fullsized":"2024-03-28T01:08:24.168298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139996+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:08:19.139996+0000","last_peered":"2024-03-28T01:08:19.139996+0000","last_clean":"2024-03-28T01:08:19.139996+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:08:19.139996+0000","last_undegraded":"2024-03-28T01:08:19.139996+0000","last_fullsized":"2024-03-28T01:08:19.139996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179379+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:08:24.179379+0000","last_peered":"2024-03-28T01:08:24.179379+0000","last_clean":"2024-03-28T01:08:24.179379+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:08:24.179379+0000","last_undegraded":"2024-03-28T01:08:24.179379+0000","last_fullsized":"2024-03-28T01:08:24.179379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582453+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:08:19.582453+0000","last_peered":"2024-03-28T01:08:19.582453+0000","last_clean":"2024-03-28T01:08:19.582453+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:08:19.582453+0000","last_undegraded":"2024-03-28T01:08:19.582453+0000","last_fullsized":"2024-03-28T01:08:19.582453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.166866+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:08:24.166866+0000","last_peered":"2024-03-28T01:08:24.166866+0000","last_clean":"2024-03-28T01:08:24.166866+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:08:24.166866+0000","last_undegraded":"2024-03-28T01:08:24.166866+0000","last_fullsized":"2024-03-28T01:08:24.166866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140107+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:08:19.140107+0000","last_peered":"2024-03-28T01:08:19.140107+0000","last_clean":"2024-03-28T01:08:19.140107+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:08:19.140107+0000","last_undegraded":"2024-03-28T01:08:19.140107+0000","last_fullsized":"2024-03-28T01:08:19.140107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:34.313143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183308+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:08:24.183308+0000","last_peered":"2024-03-28T01:08:24.183308+0000","last_clean":"2024-03-28T01:08:24.183308+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:08:24.183308+0000","last_undegraded":"2024-03-28T01:08:24.183308+0000","last_fullsized":"2024-03-28T01:08:24.183308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145990+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:08:19.145990+0000","last_peered":"2024-03-28T01:08:19.145990+0000","last_clean":"2024-03-28T01:08:19.145990+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:08:19.145990+0000","last_undegraded":"2024-03-28T01:08:19.145990+0000","last_fullsized":"2024-03-28T01:08:19.145990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171732+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:08:24.171732+0000","last_peered":"2024-03-28T01:08:24.171732+0000","last_clean":"2024-03-28T01:08:24.171732+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:08:24.171732+0000","last_undegraded":"2024-03-28T01:08:24.171732+0000","last_fullsized":"2024-03-28T01:08:24.171732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580100+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:08:19.580100+0000","last_peered":"2024-03-28T01:08:19.580100+0000","last_clean":"2024-03-28T01:08:19.580100+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:08:19.580100+0000","last_undegraded":"2024-03-28T01:08:19.580100+0000","last_fullsized":"2024-03-28T01:08:19.580100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140994+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:08:19.140994+0000","last_peered":"2024-03-28T01:08:19.140994+0000","last_clean":"2024-03-28T01:08:19.140994+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:08:19.140994+0000","last_undegraded":"2024-03-28T01:08:19.140994+0000","last_fullsized":"2024-03-28T01:08:19.140994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139443+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:08:19.139443+0000","last_peered":"2024-03-28T01:08:19.139443+0000","last_clean":"2024-03-28T01:08:19.139443+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:08:19.139443+0000","last_undegraded":"2024-03-28T01:08:19.139443+0000","last_fullsized":"2024-03-28T01:08:19.139443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168064+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:08:24.168064+0000","last_peered":"2024-03-28T01:08:24.168064+0000","last_clean":"2024-03-28T01:08:24.168064+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:08:24.168064+0000","last_undegraded":"2024-03-28T01:08:24.168064+0000","last_fullsized":"2024-03-28T01:08:24.168064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139655+0000","last_change":"2024-03-28T01:06:21.715183+0000","last_active":"2024-03-28T01:08:19.139655+0000","last_peered":"2024-03-28T01:08:19.139655+0000","last_clean":"2024-03-28T01:08:19.139655+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:08:19.139655+0000","last_undegraded":"2024-03-28T01:08:19.139655+0000","last_fullsized":"2024-03-28T01:08:19.139655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171388+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:08:24.171388+0000","last_peered":"2024-03-28T01:08:24.171388+0000","last_clean":"2024-03-28T01:08:24.171388+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:08:24.171388+0000","last_undegraded":"2024-03-28T01:08:24.171388+0000","last_fullsized":"2024-03-28T01:08:24.171388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582048+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:08:19.582048+0000","last_peered":"2024-03-28T01:08:19.582048+0000","last_clean":"2024-03-28T01:08:19.582048+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:08:19.582048+0000","last_undegraded":"2024-03-28T01:08:19.582048+0000","last_fullsized":"2024-03-28T01:08:19.582048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182659+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:08:24.182659+0000","last_peered":"2024-03-28T01:08:24.182659+0000","last_clean":"2024-03-28T01:08:24.182659+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:08:24.182659+0000","last_undegraded":"2024-03-28T01:08:24.182659+0000","last_fullsized":"2024-03-28T01:08:24.182659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'15","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182932+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:08:24.182932+0000","last_peered":"2024-03-28T01:08:24.182932+0000","last_clean":"2024-03-28T01:08:24.182932+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:08:24.182932+0000","last_undegraded":"2024-03-28T01:08:24.182932+0000","last_fullsized":"2024-03-28T01:08:24.182932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.581500+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:08:19.581500+0000","last_peered":"2024-03-28T01:08:19.581500+0000","last_clean":"2024-03-28T01:08:19.581500+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:08:19.581500+0000","last_undegraded":"2024-03-28T01:08:19.581500+0000","last_fullsized":"2024-03-28T01:08:19.581500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184715+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:08:24.184715+0000","last_peered":"2024-03-28T01:08:24.184715+0000","last_clean":"2024-03-28T01:08:24.184715+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:08:24.184715+0000","last_undegraded":"2024-03-28T01:08:24.184715+0000","last_fullsized":"2024-03-28T01:08:24.184715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582016+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:08:19.582016+0000","last_peered":"2024-03-28T01:08:19.582016+0000","last_clean":"2024-03-28T01:08:19.582016+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:08:19.582016+0000","last_undegraded":"2024-03-28T01:08:19.582016+0000","last_fullsized":"2024-03-28T01:08:19.582016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'6","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.173017+0000","last_change":"2024-03-28T01:07:03.942345+0000","last_active":"2024-03-28T01:08:24.173017+0000","last_peered":"2024-03-28T01:08:24.173017+0000","last_clean":"2024-03-28T01:08:24.173017+0000","last_became_active":"2024-03-28T01:07:03.941880+0000","last_became_peered":"2024-03-28T01:07:03.941880+0000","last_unstale":"2024-03-28T01:08:24.173017+0000","last_undegraded":"2024-03-28T01:08:24.173017+0000","last_fullsized":"2024-03-28T01:08:24.173017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:31:03.727315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.141335+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:08:19.141335+0000","last_peered":"2024-03-28T01:08:19.141335+0000","last_clean":"2024-03-28T01:08:19.141335+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:08:19.141335+0000","last_undegraded":"2024-03-28T01:08:19.141335+0000","last_fullsized":"2024-03-28T01:08:19.141335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:32.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177380+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:08:24.177380+0000","last_peered":"2024-03-28T01:08:24.177380+0000","last_clean":"2024-03-28T01:08:24.177380+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:08:24.177380+0000","last_undegraded":"2024-03-28T01:08:24.177380+0000","last_fullsized":"2024-03-28T01:08:24.177380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.579757+0000","last_change":"2024-03-28T01:06:21.727858+0000","last_active":"2024-03-28T01:08:19.579757+0000","last_peered":"2024-03-28T01:08:19.579757+0000","last_clean":"2024-03-28T01:08:19.579757+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:08:19.579757+0000","last_undegraded":"2024-03-28T01:08:19.579757+0000","last_fullsized":"2024-03-28T01:08:19.579757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:15:55.049218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":18,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":36,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":526,"ondisk_log_size":526,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":130821,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":170,"num_write_kb":213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":261642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":860160,"data_stored":680432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":218379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":89563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":89596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":397312,"data_stored":219499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":90180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":89627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":851968,"data_stored":680476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":130023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":128912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":9,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":6,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:08:29.703 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-28T01:08:29.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:08:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.953+0000 7fc04fe00640 1 Processor -- start 2024-03-28T01:08:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.953+0000 7fc04fe00640 1 -- start start 2024-03-28T01:08:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.954+0000 7fc04fe00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fc0500b51d0 0x7fc0500b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.954+0000 7fc04fe00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0500b8d00 0x7fc05014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.954+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b42f0 0x7fc05014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.954+0000 7fc04fe00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05014a820 con 0x7fc0500b8d00 2024-03-28T01:08:29.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.954+0000 7fc04fe00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0500b55b0 con 0x7fc0500b51d0 2024-03-28T01:08:29.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.954+0000 7fc04fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0500b9190 con 0x7fc0500b42f0 2024-03-28T01:08:29.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.954+0000 7fc04e000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0500b8d00 0x7fc05014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04e000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0500b8d00 0x7fc05014ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32804/0 (socket says 172.21.15.33:32804) 2024-03-28T01:08:29.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04e000640 1 -- 172.21.15.33:0/2542388607 learned_addr learned my addr 172.21.15.33:0/2542388607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04ea00640 1 --2- 172.21.15.33:0/2542388607 >> v2:172.21.15.194:3300/0 conn(0x7fc0500b51d0 0x7fc0500b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04e000640 1 -- 172.21.15.33:0/2542388607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b42f0 msgr2=0x7fc05014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04f400640 1 --2- 172.21.15.33:0/2542388607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b42f0 0x7fc05014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04e000640 1 --2- 172.21.15.33:0/2542388607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b42f0 0x7fc05014b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04e000640 1 -- 172.21.15.33:0/2542388607 >> v2:172.21.15.194:3300/0 conn(0x7fc0500b51d0 msgr2=0x7fc0500b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04e000640 1 --2- 172.21.15.33:0/2542388607 >> v2:172.21.15.194:3300/0 conn(0x7fc0500b51d0 0x7fc0500b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04e000640 1 -- 172.21.15.33:0/2542388607 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0500b9800 con 0x7fc0500b8d00 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04ea00640 1 --2- 172.21.15.33:0/2542388607 >> v2:172.21.15.194:3300/0 conn(0x7fc0500b51d0 0x7fc0500b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.955+0000 7fc04f400640 1 --2- 172.21.15.33:0/2542388607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b42f0 0x7fc05014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.956+0000 7fc04e000640 1 --2- 172.21.15.33:0/2542388607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0500b8d00 0x7fc05014ac70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0500ba2b0 tx=0x7fc03c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791d383976397199 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.957+0000 7fc04d600640 1 -- 172.21.15.33:0/2542388607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c035020 con 0x7fc0500b8d00 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.957+0000 7fc04d600640 1 -- 172.21.15.33:0/2542388607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c031870 con 0x7fc0500b8d00 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.957+0000 7fc04d600640 1 -- 172.21.15.33:0/2542388607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c031c40 con 0x7fc0500b8d00 2024-03-28T01:08:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.957+0000 7fc04fe00640 1 -- 172.21.15.33:0/2542388607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0500b8d00 msgr2=0x7fc05014ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.957+0000 7fc04fe00640 1 --2- 172.21.15.33:0/2542388607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0500b8d00 0x7fc05014ac70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0500ba2b0 tx=0x7fc03c02b910 comp rx=0 tx=0).stop 2024-03-28T01:08:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.957+0000 7fc04fe00640 1 -- 172.21.15.33:0/2542388607 shutdown_connections 2024-03-28T01:08:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.957+0000 7fc04fe00640 1 --2- 172.21.15.33:0/2542388607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b42f0 0x7fc05014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.958+0000 7fc04fe00640 1 --2- 172.21.15.33:0/2542388607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0500b8d00 0x7fc05014ac70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.958+0000 7fc04fe00640 1 --2- 172.21.15.33:0/2542388607 >> v2:172.21.15.194:3300/0 conn(0x7fc0500b51d0 0x7fc0500b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.958+0000 7fc04fe00640 1 -- 172.21.15.33:0/2542388607 >> 172.21.15.33:0/2542388607 conn(0x7fc050072940 msgr2=0x7fc050072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.958+0000 7fc04fe00640 1 -- 172.21.15.33:0/2542388607 shutdown_connections 2024-03-28T01:08:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.958+0000 7fc04fe00640 1 -- 172.21.15.33:0/2542388607 wait complete. 2024-03-28T01:08:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.961+0000 7fc04fe00640 1 Processor -- start 2024-03-28T01:08:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.961+0000 7fc04fe00640 1 -- start start 2024-03-28T01:08:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.962+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0500b42f0 0x7fc050143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.962+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b51d0 0x7fc0501dcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.962+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0501dd4e0 0x7fc05013cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.962+0000 7fc04fe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0500b39c0 con 0x7fc0501dd4e0 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.962+0000 7fc04fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05014b6f0 con 0x7fc0500b42f0 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.962+0000 7fc04fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0500b6b80 con 0x7fc0500b51d0 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0501dd4e0 0x7fc05013cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0501dd4e0 0x7fc05013cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32812/0 (socket says 172.21.15.33:32812) 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 -- 172.21.15.33:0/7698260 learned_addr learned my addr 172.21.15.33:0/7698260 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04e000640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b51d0 0x7fc0501dcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04ea00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0500b42f0 0x7fc050143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 -- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b51d0 msgr2=0x7fc0501dcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b51d0 0x7fc0501dcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 -- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0500b42f0 msgr2=0x7fc050143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0500b42f0 0x7fc050143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 -- 172.21.15.33:0/7698260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0501d1500 con 0x7fc0501dd4e0 2024-03-28T01:08:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.963+0000 7fc04f400640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0501dd4e0 0x7fc05013cb30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc04800ce80 tx=0x7fc04800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.964+0000 7fc046a00640 1 -- 172.21.15.33:0/7698260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04801a070 con 0x7fc0501dd4e0 2024-03-28T01:08:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.964+0000 7fc046a00640 1 -- 172.21.15.33:0/7698260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800d840 con 0x7fc0501dd4e0 2024-03-28T01:08:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.964+0000 7fc046a00640 1 -- 172.21.15.33:0/7698260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800dc70 con 0x7fc0501dd4e0 2024-03-28T01:08:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.964+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0500bd180 con 0x7fc0501dd4e0 2024-03-28T01:08:29.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.964+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0501d0820 con 0x7fc0501dd4e0 2024-03-28T01:08:29.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.965+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0500680b0 con 0x7fc0501dd4e0 2024-03-28T01:08:29.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.966+0000 7fc046a00640 1 -- 172.21.15.33:0/7698260 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc04801d070 con 0x7fc0501dd4e0 2024-03-28T01:08:29.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.967+0000 7fc046a00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc03006bbd0 0x7fc03006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:29.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.967+0000 7fc046a00640 1 -- 172.21.15.33:0/7698260 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fc04808b3c0 con 0x7fc0501dd4e0 2024-03-28T01:08:29.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.967+0000 7fc04ea00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc03006bbd0 0x7fc03006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:29.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.968+0000 7fc04ea00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc03006bbd0 0x7fc03006e090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc04000a910 tx=0x7fc040008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:29.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:29.972+0000 7fc046a00640 1 -- 172.21.15.33:0/7698260 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fc0500680b0 con 0x7fc0501dd4e0 2024-03-28T01:08:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.175+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc050075310 con 0x7fc03006bbd0 2024-03-28T01:08:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.190+0000 7fc046a00640 1 -- 172.21.15.33:0/7698260 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252146 (secure 0 0 0) 0x7fc050075310 con 0x7fc03006bbd0 2024-03-28T01:08:30.196 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:08:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:08:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.196+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc03006bbd0 msgr2=0x7fc03006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.196+0000 7fc04fe00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc03006bbd0 0x7fc03006e090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc04000a910 tx=0x7fc040008040 comp rx=0 tx=0).stop 2024-03-28T01:08:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.196+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0501dd4e0 msgr2=0x7fc05013cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.196+0000 7fc04fe00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0501dd4e0 0x7fc05013cb30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc04800ce80 tx=0x7fc04800a710 comp rx=0 tx=0).stop 2024-03-28T01:08:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 shutdown_connections 2024-03-28T01:08:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc03006bbd0 0x7fc03006e090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0501dd4e0 0x7fc05013cb30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0500b51d0 0x7fc0501dcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 --2- 172.21.15.33:0/7698260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0500b42f0 0x7fc050143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 >> 172.21.15.33:0/7698260 conn(0x7fc050072940 msgr2=0x7fc05013fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 shutdown_connections 2024-03-28T01:08:30.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.197+0000 7fc04fe00640 1 -- 172.21.15.33:0/7698260 wait complete. 2024-03-28T01:08:30.219 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-03-28T01:08:28.428123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":721207,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":125,"num_read_kb":109,"num_write":345,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":813,"ondisk_log_size":813,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219552,"kb_used_data":3424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755168,"statfs":{"total":773094113280,"available":772869292032,"internally_reserved":0,"allocated":3506176,"data_stored":2157752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117370,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003984"},"pg_stats":[{"pgid":"3.3f","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182684+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:08:24.182684+0000","last_peered":"2024-03-28T01:08:24.182684+0000","last_clean":"2024-03-28T01:08:24.182684+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:08:24.182684+0000","last_undegraded":"2024-03-28T01:08:24.182684+0000","last_fullsized":"2024-03-28T01:08:24.182684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'4","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.170596+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:08:24.170596+0000","last_peered":"2024-03-28T01:08:24.170596+0000","last_clean":"2024-03-28T01:08:24.170596+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:08:24.170596+0000","last_undegraded":"2024-03-28T01:08:24.170596+0000","last_fullsized":"2024-03-28T01:08:24.170596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.170497+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:08:24.170497+0000","last_peered":"2024-03-28T01:08:24.170497+0000","last_clean":"2024-03-28T01:08:24.170497+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:08:24.170497+0000","last_undegraded":"2024-03-28T01:08:24.170497+0000","last_fullsized":"2024-03-28T01:08:24.170497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184333+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:08:24.184333+0000","last_peered":"2024-03-28T01:08:24.184333+0000","last_clean":"2024-03-28T01:08:24.184333+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:08:24.184333+0000","last_undegraded":"2024-03-28T01:08:24.184333+0000","last_fullsized":"2024-03-28T01:08:24.184333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183192+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:08:24.183192+0000","last_peered":"2024-03-28T01:08:24.183192+0000","last_clean":"2024-03-28T01:08:24.183192+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:08:24.183192+0000","last_undegraded":"2024-03-28T01:08:24.183192+0000","last_fullsized":"2024-03-28T01:08:24.183192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172793+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:08:24.172793+0000","last_peered":"2024-03-28T01:08:24.172793+0000","last_clean":"2024-03-28T01:08:24.172793+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:08:24.172793+0000","last_undegraded":"2024-03-28T01:08:24.172793+0000","last_fullsized":"2024-03-28T01:08:24.172793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.166510+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:08:24.166510+0000","last_peered":"2024-03-28T01:08:24.166510+0000","last_clean":"2024-03-28T01:08:24.166510+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:08:24.166510+0000","last_undegraded":"2024-03-28T01:08:24.166510+0000","last_fullsized":"2024-03-28T01:08:24.166510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183817+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:08:24.183817+0000","last_peered":"2024-03-28T01:08:24.183817+0000","last_clean":"2024-03-28T01:08:24.183817+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:08:24.183817+0000","last_undegraded":"2024-03-28T01:08:24.183817+0000","last_fullsized":"2024-03-28T01:08:24.183817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.185365+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:08:24.185365+0000","last_peered":"2024-03-28T01:08:24.185365+0000","last_clean":"2024-03-28T01:08:24.185365+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:08:24.185365+0000","last_undegraded":"2024-03-28T01:08:24.185365+0000","last_fullsized":"2024-03-28T01:08:24.185365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'17","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184340+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:08:24.184340+0000","last_peered":"2024-03-28T01:08:24.184340+0000","last_clean":"2024-03-28T01:08:24.184340+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:08:24.184340+0000","last_undegraded":"2024-03-28T01:08:24.184340+0000","last_fullsized":"2024-03-28T01:08:24.184340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183682+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:08:24.183682+0000","last_peered":"2024-03-28T01:08:24.183682+0000","last_clean":"2024-03-28T01:08:24.183682+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:08:24.183682+0000","last_undegraded":"2024-03-28T01:08:24.183682+0000","last_fullsized":"2024-03-28T01:08:24.183682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.161730+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:08:24.161730+0000","last_peered":"2024-03-28T01:08:24.161730+0000","last_clean":"2024-03-28T01:08:24.161730+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:08:24.161730+0000","last_undegraded":"2024-03-28T01:08:24.161730+0000","last_fullsized":"2024-03-28T01:08:24.161730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182627+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:08:24.182627+0000","last_peered":"2024-03-28T01:08:24.182627+0000","last_clean":"2024-03-28T01:08:24.182627+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:08:24.182627+0000","last_undegraded":"2024-03-28T01:08:24.182627+0000","last_fullsized":"2024-03-28T01:08:24.182627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172340+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:08:24.172340+0000","last_peered":"2024-03-28T01:08:24.172340+0000","last_clean":"2024-03-28T01:08:24.172340+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:08:24.172340+0000","last_undegraded":"2024-03-28T01:08:24.172340+0000","last_fullsized":"2024-03-28T01:08:24.172340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:03:16.873591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171787+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:08:24.171787+0000","last_peered":"2024-03-28T01:08:24.171787+0000","last_clean":"2024-03-28T01:08:24.171787+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:08:24.171787+0000","last_undegraded":"2024-03-28T01:08:24.171787+0000","last_fullsized":"2024-03-28T01:08:24.171787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.173992+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:08:24.173992+0000","last_peered":"2024-03-28T01:08:24.173992+0000","last_clean":"2024-03-28T01:08:24.173992+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:08:24.173992+0000","last_undegraded":"2024-03-28T01:08:24.173992+0000","last_fullsized":"2024-03-28T01:08:24.173992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.176879+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:08:24.176879+0000","last_peered":"2024-03-28T01:08:24.176879+0000","last_clean":"2024-03-28T01:08:24.176879+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:08:24.176879+0000","last_undegraded":"2024-03-28T01:08:24.176879+0000","last_fullsized":"2024-03-28T01:08:24.176879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179164+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:08:24.179164+0000","last_peered":"2024-03-28T01:08:24.179164+0000","last_clean":"2024-03-28T01:08:24.179164+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:08:24.179164+0000","last_undegraded":"2024-03-28T01:08:24.179164+0000","last_fullsized":"2024-03-28T01:08:24.179164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'7","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.169769+0000","last_change":"2024-03-28T01:07:03.939021+0000","last_active":"2024-03-28T01:08:24.169769+0000","last_peered":"2024-03-28T01:08:24.169769+0000","last_clean":"2024-03-28T01:08:24.169769+0000","last_became_active":"2024-03-28T01:07:03.937565+0000","last_became_peered":"2024-03-28T01:07:03.937565+0000","last_unstale":"2024-03-28T01:08:24.169769+0000","last_undegraded":"2024-03-28T01:08:24.169769+0000","last_fullsized":"2024-03-28T01:08:24.169769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:13:26.015917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181877+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:08:24.181877+0000","last_peered":"2024-03-28T01:08:24.181877+0000","last_clean":"2024-03-28T01:08:24.181877+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:08:24.181877+0000","last_undegraded":"2024-03-28T01:08:24.181877+0000","last_fullsized":"2024-03-28T01:08:24.181877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.178531+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:08:24.178531+0000","last_peered":"2024-03-28T01:08:24.178531+0000","last_clean":"2024-03-28T01:08:24.178531+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:08:24.178531+0000","last_undegraded":"2024-03-28T01:08:24.178531+0000","last_fullsized":"2024-03-28T01:08:24.178531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:32:42.370031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183967+0000","last_change":"2024-03-28T01:07:03.972650+0000","last_active":"2024-03-28T01:08:24.183967+0000","last_peered":"2024-03-28T01:08:24.183967+0000","last_clean":"2024-03-28T01:08:24.183967+0000","last_became_active":"2024-03-28T01:07:03.972014+0000","last_became_peered":"2024-03-28T01:07:03.972014+0000","last_unstale":"2024-03-28T01:08:24.183967+0000","last_undegraded":"2024-03-28T01:08:24.183967+0000","last_fullsized":"2024-03-28T01:08:24.183967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172851+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:08:24.172851+0000","last_peered":"2024-03-28T01:08:24.172851+0000","last_clean":"2024-03-28T01:08:24.172851+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:08:24.172851+0000","last_undegraded":"2024-03-28T01:08:24.172851+0000","last_fullsized":"2024-03-28T01:08:24.172851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183405+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:08:24.183405+0000","last_peered":"2024-03-28T01:08:24.183405+0000","last_clean":"2024-03-28T01:08:24.183405+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:08:24.183405+0000","last_undegraded":"2024-03-28T01:08:24.183405+0000","last_fullsized":"2024-03-28T01:08:24.183405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168319+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:08:24.168319+0000","last_peered":"2024-03-28T01:08:24.168319+0000","last_clean":"2024-03-28T01:08:24.168319+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:08:24.168319+0000","last_undegraded":"2024-03-28T01:08:24.168319+0000","last_fullsized":"2024-03-28T01:08:24.168319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'16","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180766+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:08:24.180766+0000","last_peered":"2024-03-28T01:08:24.180766+0000","last_clean":"2024-03-28T01:08:24.180766+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:08:24.180766+0000","last_undegraded":"2024-03-28T01:08:24.180766+0000","last_fullsized":"2024-03-28T01:08:24.180766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177548+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:08:24.177548+0000","last_peered":"2024-03-28T01:08:24.177548+0000","last_clean":"2024-03-28T01:08:24.177548+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:08:24.177548+0000","last_undegraded":"2024-03-28T01:08:24.177548+0000","last_fullsized":"2024-03-28T01:08:24.177548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'4","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182503+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:08:24.182503+0000","last_peered":"2024-03-28T01:08:24.182503+0000","last_clean":"2024-03-28T01:08:24.182503+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:08:24.182503+0000","last_undegraded":"2024-03-28T01:08:24.182503+0000","last_fullsized":"2024-03-28T01:08:24.182503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'7","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.172868+0000","last_change":"2024-03-28T01:07:03.942155+0000","last_active":"2024-03-28T01:08:24.172868+0000","last_peered":"2024-03-28T01:08:24.172868+0000","last_clean":"2024-03-28T01:08:24.172868+0000","last_became_active":"2024-03-28T01:07:03.941729+0000","last_became_peered":"2024-03-28T01:07:03.941729+0000","last_unstale":"2024-03-28T01:08:24.172868+0000","last_undegraded":"2024-03-28T01:08:24.172868+0000","last_fullsized":"2024-03-28T01:08:24.172868+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:13:26.015917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'8","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.158440+0000","last_change":"2024-03-28T01:07:03.930604+0000","last_active":"2024-03-28T01:08:24.158440+0000","last_peered":"2024-03-28T01:08:24.158440+0000","last_clean":"2024-03-28T01:08:24.158440+0000","last_became_active":"2024-03-28T01:07:03.929334+0000","last_became_peered":"2024-03-28T01:07:03.929334+0000","last_unstale":"2024-03-28T01:08:24.158440+0000","last_undegraded":"2024-03-28T01:08:24.158440+0000","last_fullsized":"2024-03-28T01:08:24.158440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179921+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:08:24.179921+0000","last_peered":"2024-03-28T01:08:24.179921+0000","last_clean":"2024-03-28T01:08:24.179921+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:08:24.179921+0000","last_undegraded":"2024-03-28T01:08:24.179921+0000","last_fullsized":"2024-03-28T01:08:24.179921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:35.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180248+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:08:24.180248+0000","last_peered":"2024-03-28T01:08:24.180248+0000","last_clean":"2024-03-28T01:08:24.180248+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:08:24.180248+0000","last_undegraded":"2024-03-28T01:08:24.180248+0000","last_fullsized":"2024-03-28T01:08:24.180248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.152922+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:08:24.152922+0000","last_peered":"2024-03-28T01:08:24.152922+0000","last_clean":"2024-03-28T01:08:24.152922+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:08:24.152922+0000","last_undegraded":"2024-03-28T01:08:24.152922+0000","last_fullsized":"2024-03-28T01:08:24.152922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140953+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:08:19.140953+0000","last_peered":"2024-03-28T01:08:19.140953+0000","last_clean":"2024-03-28T01:08:19.140953+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:08:19.140953+0000","last_undegraded":"2024-03-28T01:08:19.140953+0000","last_fullsized":"2024-03-28T01:08:19.140953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181010+0000","last_change":"2024-03-28T01:06:22.717986+0000","last_active":"2024-03-28T01:08:24.181010+0000","last_peered":"2024-03-28T01:08:24.181010+0000","last_clean":"2024-03-28T01:08:24.181010+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:08:24.181010+0000","last_undegraded":"2024-03-28T01:08:24.181010+0000","last_fullsized":"2024-03-28T01:08:24.181010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145219+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:08:19.145219+0000","last_peered":"2024-03-28T01:08:19.145219+0000","last_clean":"2024-03-28T01:08:19.145219+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:08:19.145219+0000","last_undegraded":"2024-03-28T01:08:19.145219+0000","last_fullsized":"2024-03-28T01:08:19.145219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140916+0000","last_change":"2024-03-28T01:06:21.721278+0000","last_active":"2024-03-28T01:08:19.140916+0000","last_peered":"2024-03-28T01:08:19.140916+0000","last_clean":"2024-03-28T01:08:19.140916+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:08:19.140916+0000","last_undegraded":"2024-03-28T01:08:19.140916+0000","last_fullsized":"2024-03-28T01:08:19.140916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177391+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:08:24.177391+0000","last_peered":"2024-03-28T01:08:24.177391+0000","last_clean":"2024-03-28T01:08:24.177391+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:08:24.177391+0000","last_undegraded":"2024-03-28T01:08:24.177391+0000","last_fullsized":"2024-03-28T01:08:24.177391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580460+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:08:19.580460+0000","last_peered":"2024-03-28T01:08:19.580460+0000","last_clean":"2024-03-28T01:08:19.580460+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:08:19.580460+0000","last_undegraded":"2024-03-28T01:08:19.580460+0000","last_fullsized":"2024-03-28T01:08:19.580460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.178018+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:08:24.178018+0000","last_peered":"2024-03-28T01:08:24.178018+0000","last_clean":"2024-03-28T01:08:24.178018+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:08:24.178018+0000","last_undegraded":"2024-03-28T01:08:24.178018+0000","last_fullsized":"2024-03-28T01:08:24.178018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145869+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:08:19.145869+0000","last_peered":"2024-03-28T01:08:19.145869+0000","last_clean":"2024-03-28T01:08:19.145869+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:08:19.145869+0000","last_undegraded":"2024-03-28T01:08:19.145869+0000","last_fullsized":"2024-03-28T01:08:19.145869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184253+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:08:24.184253+0000","last_peered":"2024-03-28T01:08:24.184253+0000","last_clean":"2024-03-28T01:08:24.184253+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:08:24.184253+0000","last_undegraded":"2024-03-28T01:08:24.184253+0000","last_fullsized":"2024-03-28T01:08:24.184253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582176+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:08:19.582176+0000","last_peered":"2024-03-28T01:08:19.582176+0000","last_clean":"2024-03-28T01:08:19.582176+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:08:19.582176+0000","last_undegraded":"2024-03-28T01:08:19.582176+0000","last_fullsized":"2024-03-28T01:08:19.582176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'8","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.185188+0000","last_change":"2024-03-28T01:07:03.970890+0000","last_active":"2024-03-28T01:08:24.185188+0000","last_peered":"2024-03-28T01:08:24.185188+0000","last_clean":"2024-03-28T01:08:24.185188+0000","last_became_active":"2024-03-28T01:07:03.970312+0000","last_became_peered":"2024-03-28T01:07:03.970312+0000","last_unstale":"2024-03-28T01:08:24.185188+0000","last_undegraded":"2024-03-28T01:08:24.185188+0000","last_fullsized":"2024-03-28T01:08:24.185188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139870+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:08:19.139870+0000","last_peered":"2024-03-28T01:08:19.139870+0000","last_clean":"2024-03-28T01:08:19.139870+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:08:19.139870+0000","last_undegraded":"2024-03-28T01:08:19.139870+0000","last_fullsized":"2024-03-28T01:08:19.139870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179615+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:08:24.179615+0000","last_peered":"2024-03-28T01:08:24.179615+0000","last_clean":"2024-03-28T01:08:24.179615+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:08:24.179615+0000","last_undegraded":"2024-03-28T01:08:24.179615+0000","last_fullsized":"2024-03-28T01:08:24.179615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'111","reported_seq":131,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.200095+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:08:24.200095+0000","last_peered":"2024-03-28T01:08:24.200095+0000","last_clean":"2024-03-28T01:08:24.200095+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:08:24.200095+0000","last_undegraded":"2024-03-28T01:08:24.200095+0000","last_fullsized":"2024-03-28T01:08:24.200095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.193557+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:08:24.193557+0000","last_peered":"2024-03-28T01:08:24.193557+0000","last_clean":"2024-03-28T01:08:24.193557+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:08:24.193557+0000","last_undegraded":"2024-03-28T01:08:24.193557+0000","last_fullsized":"2024-03-28T01:08:24.193557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580187+0000","last_change":"2024-03-28T01:06:21.727305+0000","last_active":"2024-03-28T01:08:19.580187+0000","last_peered":"2024-03-28T01:08:19.580187+0000","last_clean":"2024-03-28T01:08:19.580187+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:08:19.580187+0000","last_undegraded":"2024-03-28T01:08:19.580187+0000","last_fullsized":"2024-03-28T01:08:19.580187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'11","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179008+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:08:24.179008+0000","last_peered":"2024-03-28T01:08:24.179008+0000","last_clean":"2024-03-28T01:08:24.179008+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:08:24.179008+0000","last_undegraded":"2024-03-28T01:08:24.179008+0000","last_fullsized":"2024-03-28T01:08:24.179008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:57:31.424544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580799+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:08:19.580799+0000","last_peered":"2024-03-28T01:08:19.580799+0000","last_clean":"2024-03-28T01:08:19.580799+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:08:19.580799+0000","last_undegraded":"2024-03-28T01:08:19.580799+0000","last_fullsized":"2024-03-28T01:08:19.580799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'15","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183597+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:08:24.183597+0000","last_peered":"2024-03-28T01:08:24.183597+0000","last_clean":"2024-03-28T01:08:24.183597+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:08:24.183597+0000","last_undegraded":"2024-03-28T01:08:24.183597+0000","last_fullsized":"2024-03-28T01:08:24.183597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145073+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:08:19.145073+0000","last_peered":"2024-03-28T01:08:19.145073+0000","last_clean":"2024-03-28T01:08:19.145073+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:08:19.145073+0000","last_undegraded":"2024-03-28T01:08:19.145073+0000","last_fullsized":"2024-03-28T01:08:19.145073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.158153+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:08:24.158153+0000","last_peered":"2024-03-28T01:08:24.158153+0000","last_clean":"2024-03-28T01:08:24.158153+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:08:24.158153+0000","last_undegraded":"2024-03-28T01:08:24.158153+0000","last_fullsized":"2024-03-28T01:08:24.158153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.579911+0000","last_change":"2024-03-28T01:06:21.719466+0000","last_active":"2024-03-28T01:08:19.579911+0000","last_peered":"2024-03-28T01:08:19.579911+0000","last_clean":"2024-03-28T01:08:19.579911+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:08:19.579911+0000","last_undegraded":"2024-03-28T01:08:19.579911+0000","last_fullsized":"2024-03-28T01:08:19.579911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:04:37.987556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181084+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:08:24.181084+0000","last_peered":"2024-03-28T01:08:24.181084+0000","last_clean":"2024-03-28T01:08:24.181084+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:08:24.181084+0000","last_undegraded":"2024-03-28T01:08:24.181084+0000","last_fullsized":"2024-03-28T01:08:24.181084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.142534+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:08:19.142534+0000","last_peered":"2024-03-28T01:08:19.142534+0000","last_clean":"2024-03-28T01:08:19.142534+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:08:19.142534+0000","last_undegraded":"2024-03-28T01:08:19.142534+0000","last_fullsized":"2024-03-28T01:08:19.142534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181248+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:08:24.181248+0000","last_peered":"2024-03-28T01:08:24.181248+0000","last_clean":"2024-03-28T01:08:24.181248+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:08:24.181248+0000","last_undegraded":"2024-03-28T01:08:24.181248+0000","last_fullsized":"2024-03-28T01:08:24.181248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580856+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:08:19.580856+0000","last_peered":"2024-03-28T01:08:19.580856+0000","last_clean":"2024-03-28T01:08:19.580856+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:08:19.580856+0000","last_undegraded":"2024-03-28T01:08:19.580856+0000","last_fullsized":"2024-03-28T01:08:19.580856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183144+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:08:24.183144+0000","last_peered":"2024-03-28T01:08:24.183144+0000","last_clean":"2024-03-28T01:08:24.183144+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:08:24.183144+0000","last_undegraded":"2024-03-28T01:08:24.183144+0000","last_fullsized":"2024-03-28T01:08:24.183144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139941+0000","last_change":"2024-03-28T01:06:21.718159+0000","last_active":"2024-03-28T01:08:19.139941+0000","last_peered":"2024-03-28T01:08:19.139941+0000","last_clean":"2024-03-28T01:08:19.139941+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:08:19.139941+0000","last_undegraded":"2024-03-28T01:08:19.139941+0000","last_fullsized":"2024-03-28T01:08:19.139941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182471+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:08:24.182471+0000","last_peered":"2024-03-28T01:08:24.182471+0000","last_clean":"2024-03-28T01:08:24.182471+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:08:24.182471+0000","last_undegraded":"2024-03-28T01:08:24.182471+0000","last_fullsized":"2024-03-28T01:08:24.182471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580700+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:08:19.580700+0000","last_peered":"2024-03-28T01:08:19.580700+0000","last_clean":"2024-03-28T01:08:19.580700+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:08:19.580700+0000","last_undegraded":"2024-03-28T01:08:19.580700+0000","last_fullsized":"2024-03-28T01:08:19.580700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182881+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:08:24.182881+0000","last_peered":"2024-03-28T01:08:24.182881+0000","last_clean":"2024-03-28T01:08:24.182881+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:08:24.182881+0000","last_undegraded":"2024-03-28T01:08:24.182881+0000","last_fullsized":"2024-03-28T01:08:24.182881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580583+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:08:19.580583+0000","last_peered":"2024-03-28T01:08:19.580583+0000","last_clean":"2024-03-28T01:08:19.580583+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:08:19.580583+0000","last_undegraded":"2024-03-28T01:08:19.580583+0000","last_fullsized":"2024-03-28T01:08:19.580583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580787+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:08:19.580787+0000","last_peered":"2024-03-28T01:08:19.580787+0000","last_clean":"2024-03-28T01:08:19.580787+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:08:19.580787+0000","last_undegraded":"2024-03-28T01:08:19.580787+0000","last_fullsized":"2024-03-28T01:08:19.580787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.181907+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:08:24.181907+0000","last_peered":"2024-03-28T01:08:24.181907+0000","last_clean":"2024-03-28T01:08:24.181907+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:08:24.181907+0000","last_undegraded":"2024-03-28T01:08:24.181907+0000","last_fullsized":"2024-03-28T01:08:24.181907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580554+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:08:19.580554+0000","last_peered":"2024-03-28T01:08:19.580554+0000","last_clean":"2024-03-28T01:08:19.580554+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:08:19.580554+0000","last_undegraded":"2024-03-28T01:08:19.580554+0000","last_fullsized":"2024-03-28T01:08:19.580554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.180800+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:08:24.180800+0000","last_peered":"2024-03-28T01:08:24.180800+0000","last_clean":"2024-03-28T01:08:24.180800+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:08:24.180800+0000","last_undegraded":"2024-03-28T01:08:24.180800+0000","last_fullsized":"2024-03-28T01:08:24.180800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:30.912521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'10","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.886078+0000","last_change":"2024-03-28T01:06:21.720753+0000","last_active":"2024-03-28T01:08:24.886078+0000","last_peered":"2024-03-28T01:08:24.886078+0000","last_clean":"2024-03-28T01:08:24.886078+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:08:24.886078+0000","last_undegraded":"2024-03-28T01:08:24.886078+0000","last_fullsized":"2024-03-28T01:08:24.886078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.161990+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:08:24.161990+0000","last_peered":"2024-03-28T01:08:24.161990+0000","last_clean":"2024-03-28T01:08:24.161990+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:08:24.161990+0000","last_undegraded":"2024-03-28T01:08:24.161990+0000","last_fullsized":"2024-03-28T01:08:24.161990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.581181+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:08:19.581181+0000","last_peered":"2024-03-28T01:08:19.581181+0000","last_clean":"2024-03-28T01:08:19.581181+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:08:19.581181+0000","last_undegraded":"2024-03-28T01:08:19.581181+0000","last_fullsized":"2024-03-28T01:08:19.581181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168298+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:08:24.168298+0000","last_peered":"2024-03-28T01:08:24.168298+0000","last_clean":"2024-03-28T01:08:24.168298+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:08:24.168298+0000","last_undegraded":"2024-03-28T01:08:24.168298+0000","last_fullsized":"2024-03-28T01:08:24.168298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139996+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:08:19.139996+0000","last_peered":"2024-03-28T01:08:19.139996+0000","last_clean":"2024-03-28T01:08:19.139996+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:08:19.139996+0000","last_undegraded":"2024-03-28T01:08:19.139996+0000","last_fullsized":"2024-03-28T01:08:19.139996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.179379+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:08:24.179379+0000","last_peered":"2024-03-28T01:08:24.179379+0000","last_clean":"2024-03-28T01:08:24.179379+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:08:24.179379+0000","last_undegraded":"2024-03-28T01:08:24.179379+0000","last_fullsized":"2024-03-28T01:08:24.179379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582453+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:08:19.582453+0000","last_peered":"2024-03-28T01:08:19.582453+0000","last_clean":"2024-03-28T01:08:19.582453+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:08:19.582453+0000","last_undegraded":"2024-03-28T01:08:19.582453+0000","last_fullsized":"2024-03-28T01:08:19.582453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.166866+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:08:24.166866+0000","last_peered":"2024-03-28T01:08:24.166866+0000","last_clean":"2024-03-28T01:08:24.166866+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:08:24.166866+0000","last_undegraded":"2024-03-28T01:08:24.166866+0000","last_fullsized":"2024-03-28T01:08:24.166866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140107+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:08:19.140107+0000","last_peered":"2024-03-28T01:08:19.140107+0000","last_clean":"2024-03-28T01:08:19.140107+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:08:19.140107+0000","last_undegraded":"2024-03-28T01:08:19.140107+0000","last_fullsized":"2024-03-28T01:08:19.140107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:01:34.313143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.183308+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:08:24.183308+0000","last_peered":"2024-03-28T01:08:24.183308+0000","last_clean":"2024-03-28T01:08:24.183308+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:08:24.183308+0000","last_undegraded":"2024-03-28T01:08:24.183308+0000","last_fullsized":"2024-03-28T01:08:24.183308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145990+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:08:19.145990+0000","last_peered":"2024-03-28T01:08:19.145990+0000","last_clean":"2024-03-28T01:08:19.145990+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:08:19.145990+0000","last_undegraded":"2024-03-28T01:08:19.145990+0000","last_fullsized":"2024-03-28T01:08:19.145990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'7","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171732+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:08:24.171732+0000","last_peered":"2024-03-28T01:08:24.171732+0000","last_clean":"2024-03-28T01:08:24.171732+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:08:24.171732+0000","last_undegraded":"2024-03-28T01:08:24.171732+0000","last_fullsized":"2024-03-28T01:08:24.171732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580100+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:08:19.580100+0000","last_peered":"2024-03-28T01:08:19.580100+0000","last_clean":"2024-03-28T01:08:19.580100+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:08:19.580100+0000","last_undegraded":"2024-03-28T01:08:19.580100+0000","last_fullsized":"2024-03-28T01:08:19.580100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.140994+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:08:19.140994+0000","last_peered":"2024-03-28T01:08:19.140994+0000","last_clean":"2024-03-28T01:08:19.140994+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:08:19.140994+0000","last_undegraded":"2024-03-28T01:08:19.140994+0000","last_fullsized":"2024-03-28T01:08:19.140994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139443+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:08:19.139443+0000","last_peered":"2024-03-28T01:08:19.139443+0000","last_clean":"2024-03-28T01:08:19.139443+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:08:19.139443+0000","last_undegraded":"2024-03-28T01:08:19.139443+0000","last_fullsized":"2024-03-28T01:08:19.139443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.168064+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:08:24.168064+0000","last_peered":"2024-03-28T01:08:24.168064+0000","last_clean":"2024-03-28T01:08:24.168064+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:08:24.168064+0000","last_undegraded":"2024-03-28T01:08:24.168064+0000","last_fullsized":"2024-03-28T01:08:24.168064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.139655+0000","last_change":"2024-03-28T01:06:21.715183+0000","last_active":"2024-03-28T01:08:19.139655+0000","last_peered":"2024-03-28T01:08:19.139655+0000","last_clean":"2024-03-28T01:08:19.139655+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:08:19.139655+0000","last_undegraded":"2024-03-28T01:08:19.139655+0000","last_fullsized":"2024-03-28T01:08:19.139655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'8","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.171388+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:08:24.171388+0000","last_peered":"2024-03-28T01:08:24.171388+0000","last_clean":"2024-03-28T01:08:24.171388+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:08:24.171388+0000","last_undegraded":"2024-03-28T01:08:24.171388+0000","last_fullsized":"2024-03-28T01:08:24.171388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582048+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:08:19.582048+0000","last_peered":"2024-03-28T01:08:19.582048+0000","last_clean":"2024-03-28T01:08:19.582048+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:08:19.582048+0000","last_undegraded":"2024-03-28T01:08:19.582048+0000","last_fullsized":"2024-03-28T01:08:19.582048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182659+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:08:24.182659+0000","last_peered":"2024-03-28T01:08:24.182659+0000","last_clean":"2024-03-28T01:08:24.182659+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:08:24.182659+0000","last_undegraded":"2024-03-28T01:08:24.182659+0000","last_fullsized":"2024-03-28T01:08:24.182659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'15","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.182932+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:08:24.182932+0000","last_peered":"2024-03-28T01:08:24.182932+0000","last_clean":"2024-03-28T01:08:24.182932+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:08:24.182932+0000","last_undegraded":"2024-03-28T01:08:24.182932+0000","last_fullsized":"2024-03-28T01:08:24.182932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.581500+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:08:19.581500+0000","last_peered":"2024-03-28T01:08:19.581500+0000","last_clean":"2024-03-28T01:08:19.581500+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:08:19.581500+0000","last_undegraded":"2024-03-28T01:08:19.581500+0000","last_fullsized":"2024-03-28T01:08:19.581500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'9","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.184715+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:08:24.184715+0000","last_peered":"2024-03-28T01:08:24.184715+0000","last_clean":"2024-03-28T01:08:24.184715+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:08:24.184715+0000","last_undegraded":"2024-03-28T01:08:24.184715+0000","last_fullsized":"2024-03-28T01:08:24.184715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582016+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:08:19.582016+0000","last_peered":"2024-03-28T01:08:19.582016+0000","last_clean":"2024-03-28T01:08:19.582016+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:08:19.582016+0000","last_undegraded":"2024-03-28T01:08:19.582016+0000","last_fullsized":"2024-03-28T01:08:19.582016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'6","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.173017+0000","last_change":"2024-03-28T01:07:03.942345+0000","last_active":"2024-03-28T01:08:24.173017+0000","last_peered":"2024-03-28T01:08:24.173017+0000","last_clean":"2024-03-28T01:08:24.173017+0000","last_became_active":"2024-03-28T01:07:03.941880+0000","last_became_peered":"2024-03-28T01:07:03.941880+0000","last_unstale":"2024-03-28T01:08:24.173017+0000","last_undegraded":"2024-03-28T01:08:24.173017+0000","last_fullsized":"2024-03-28T01:08:24.173017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:31:03.727315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.141335+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:08:19.141335+0000","last_peered":"2024-03-28T01:08:19.141335+0000","last_clean":"2024-03-28T01:08:19.141335+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:08:19.141335+0000","last_undegraded":"2024-03-28T01:08:19.141335+0000","last_fullsized":"2024-03-28T01:08:19.141335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:32.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.177380+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:08:24.177380+0000","last_peered":"2024-03-28T01:08:24.177380+0000","last_clean":"2024-03-28T01:08:24.177380+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:08:24.177380+0000","last_undegraded":"2024-03-28T01:08:24.177380+0000","last_fullsized":"2024-03-28T01:08:24.177380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.579757+0000","last_change":"2024-03-28T01:06:21.727858+0000","last_active":"2024-03-28T01:08:19.579757+0000","last_peered":"2024-03-28T01:08:19.579757+0000","last_clean":"2024-03-28T01:08:19.579757+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:08:19.579757+0000","last_undegraded":"2024-03-28T01:08:19.579757+0000","last_fullsized":"2024-03-28T01:08:19.579757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:15:55.049218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":18,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":36,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":526,"ondisk_log_size":526,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":130821,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":170,"num_write_kb":213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":261642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":860160,"data_stored":680432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":218379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":89563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":89596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":397312,"data_stored":219499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":90180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":89627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":851968,"data_stored":680476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":130023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":128912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":9,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":6,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:08:30.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:08:30.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.505+0000 7f8f8b200640 1 Processor -- start 2024-03-28T01:08:30.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.506+0000 7f8f8b200640 1 -- start start 2024-03-28T01:08:30.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c0b6f60 0x7f8f8c13ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:30.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8b200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8f8c1485b0 0x7f8f8c13f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:30.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8b200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8c13f710 con 0x7f8f8c0b7b60 2024-03-28T01:08:30.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8b200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b7f40 con 0x7f8f8c1485b0 2024-03-28T01:08:30.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b23a0 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8a800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8f8c1485b0 0x7f8f8c13f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:30.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8a800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8f8c1485b0 0x7f8f8c13f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32818/0 (socket says 172.21.15.33:32818) 2024-03-28T01:08:30.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.507+0000 7f8f8a800640 1 -- 172.21.15.33:0/4293528445 learned_addr learned my addr 172.21.15.33:0/4293528445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:30.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.508+0000 7f8f89400640 1 --2- 172.21.15.33:0/4293528445 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c0b6f60 0x7f8f8c13ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.508+0000 7f8f8a800640 1 -- 172.21.15.33:0/4293528445 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c0b6f60 msgr2=0x7f8f8c13ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.508+0000 7f8f8a800640 1 --2- 172.21.15.33:0/4293528445 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c0b6f60 0x7f8f8c13ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.508+0000 7f8f8a800640 1 -- 172.21.15.33:0/4293528445 >> v1:172.21.15.194:6789/0 conn(0x7f8f8c0b7b60 legacy=0x7f8f8c072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.508+0000 7f8f8a800640 1 -- 172.21.15.33:0/4293528445 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c071bb0 con 0x7f8f8c1485b0 2024-03-28T01:08:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.509+0000 7f8f8a800640 1 --2- 172.21.15.33:0/4293528445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c1485b0 0x7f8f8c13f1d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009e50 tx=0x7f8f8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c828796c939252 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.509+0000 7f8f88a00640 1 -- 172.21.15.33:0/4293528445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8402ca70 con 0x7f8f8c1485b0 2024-03-28T01:08:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.509+0000 7f8f88a00640 1 -- 172.21.15.33:0/4293528445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8402cc10 con 0x7f8f8c1485b0 2024-03-28T01:08:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.509+0000 7f8f88a00640 1 -- 172.21.15.33:0/4293528445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f840343f0 con 0x7f8f8c1485b0 2024-03-28T01:08:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 -- 172.21.15.33:0/4293528445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c1485b0 msgr2=0x7f8f8c13f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4293528445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c1485b0 0x7f8f8c13f1d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009e50 tx=0x7f8f8402bed0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 -- 172.21.15.33:0/4293528445 shutdown_connections 2024-03-28T01:08:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4293528445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c1485b0 0x7f8f8c13f1d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4293528445 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c0b6f60 0x7f8f8c13ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 -- 172.21.15.33:0/4293528445 >> 172.21.15.33:0/4293528445 conn(0x7f8f8c06b450 msgr2=0x7f8f8c06b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 -- 172.21.15.33:0/4293528445 shutdown_connections 2024-03-28T01:08:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.510+0000 7f8f8b200640 1 -- 172.21.15.33:0/4293528445 wait complete. 2024-03-28T01:08:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.512+0000 7f8f8b200640 1 Processor -- start 2024-03-28T01:08:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.512+0000 7f8f8b200640 1 -- start start 2024-03-28T01:08:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f8b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0b6f60 0x7f8f8c1465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f8b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f8c0b7b60 0x7f8f8c1dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f8b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c1485b0 0x7f8f8c138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f8b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b4ca0 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f8b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b2000 con 0x7f8f8c0b7b60 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f8b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b4e20 con 0x7f8f8c1485b0 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0b6f60 0x7f8f8c1465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0b6f60 0x7f8f8c1465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32824/0 (socket says 172.21.15.33:32824) 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89e00640 1 -- 172.21.15.33:0/4013520112 learned_addr learned my addr 172.21.15.33:0/4013520112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89e00640 1 -- 172.21.15.33:0/4013520112 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c1485b0 msgr2=0x7f8f8c138ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89e00640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c1485b0 0x7f8f8c138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89e00640 1 -- 172.21.15.33:0/4013520112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f8c0b7b60 msgr2=0x7f8f8c1dcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89400640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f8c0b7b60 0x7f8f8c1dcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.513+0000 7f8f89e00640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f8c0b7b60 0x7f8f8c1dcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.514+0000 7f8f89e00640 1 -- 172.21.15.33:0/4013520112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c06dc70 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.514+0000 7f8f89e00640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0b6f60 0x7f8f8c1465a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00c430 tx=0x7f8f7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.514+0000 7f8f82000640 1 -- 172.21.15.33:0/4013520112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c017070 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.514+0000 7f8f82000640 1 -- 172.21.15.33:0/4013520112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0052d0 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.514+0000 7f8f82000640 1 -- 172.21.15.33:0/4013520112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c003850 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.514+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c147c10 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.514+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c06d470 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.515+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c072fd0 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.516+0000 7f8f82000640 1 -- 172.21.15.33:0/4013520112 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8f7c01a070 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.518+0000 7f8f82000640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8f6806bbd0 0x7f8f6806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.518+0000 7f8f82000640 1 -- 172.21.15.33:0/4013520112 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f8f7c089f20 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.518+0000 7f8f89400640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8f6806bbd0 0x7f8f6806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.519+0000 7f8f89400640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8f6806bbd0 0x7f8f6806e090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c06c9c0 tx=0x7f8f74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.525+0000 7f8f82000640 1 -- 172.21.15.33:0/4013520112 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8f8c072fd0 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.728+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f8c075310 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.731+0000 7f8f82000640 1 -- 172.21.15.33:0/4013520112 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f8f7c058920 con 0x7f8f8c0b6f60 2024-03-28T01:08:30.734 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:08:30.734 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:08:19.131295+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/4184856115":"2024-03-28T02:08:18.832612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:08:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8f6806bbd0 msgr2=0x7f8f6806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8f6806bbd0 0x7f8f6806e090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c06c9c0 tx=0x7f8f74008040 comp rx=0 tx=0).stop 2024-03-28T01:08:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0b6f60 msgr2=0x7f8f8c1465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0b6f60 0x7f8f8c1465a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00c430 tx=0x7f8f7c00c8f0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 shutdown_connections 2024-03-28T01:08:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f8f6806bbd0 0x7f8f6806e090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f8c1485b0 0x7f8f8c138ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f8c0b7b60 0x7f8f8c1dcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 --2- 172.21.15.33:0/4013520112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0b6f60 0x7f8f8c1465a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 >> 172.21.15.33:0/4013520112 conn(0x7f8f8c06b450 msgr2=0x7f8f8c145960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 shutdown_connections 2024-03-28T01:08:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:30.734+0000 7f8f8b200640 1 -- 172.21.15.33:0/4013520112 wait complete. 2024-03-28T01:08:31.759 INFO:tasks.ceph:Scrubbing osd.0 2024-03-28T01:08:31.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.982+0000 7f7d41800640 1 Processor -- start 2024-03-28T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.983+0000 7f7d41800640 1 -- start start 2024-03-28T01:08:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.984+0000 7f7d41800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7d3c0b53d0 0x7f7d3c14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.984+0000 7f7d41800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d3c0b3f90 0x7f7d3c140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.984+0000 7f7d41800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c0b4710 con 0x7f7d3c14ad00 2024-03-28T01:08:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.984+0000 7f7d41800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c072ea0 con 0x7f7d3c0b3f90 2024-03-28T01:08:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.984+0000 7f7d41800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c072b30 con 0x7f7d3c0b53d0 2024-03-28T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.985+0000 7f7d3b400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d3c0b3f90 0x7f7d3c140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.985+0000 7f7d40e00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7d3c14ad00 0x7f7d3c0b74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43648/0 (socket says 172.21.15.33:43648) 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.985+0000 7f7d40e00640 1 -- 172.21.15.33:0/1179559718 learned_addr learned my addr 172.21.15.33:0/1179559718 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.985+0000 7f7d3be00640 1 --2- 172.21.15.33:0/1179559718 >> v2:172.21.15.194:3300/0 conn(0x7f7d3c0b53d0 0x7f7d3c14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.985+0000 7f7d3aa00640 1 -- 172.21.15.33:0/1179559718 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472405557 0 0) 0x7f7d3c0b4710 con 0x7f7d3c14ad00 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.985+0000 7f7d3aa00640 1 -- 172.21.15.33:0/1179559718 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d20003520 con 0x7f7d3c14ad00 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.986+0000 7f7d3b400640 1 -- 172.21.15.33:0/1179559718 >> v2:172.21.15.194:3300/0 conn(0x7f7d3c0b53d0 msgr2=0x7f7d3c14a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.986+0000 7f7d3b400640 1 --2- 172.21.15.33:0/1179559718 >> v2:172.21.15.194:3300/0 conn(0x7f7d3c0b53d0 0x7f7d3c14a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.986+0000 7f7d3b400640 1 -- 172.21.15.33:0/1179559718 >> v1:172.21.15.33:6789/0 conn(0x7f7d3c14ad00 legacy=0x7f7d3c0b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.986+0000 7f7d3b400640 1 -- 172.21.15.33:0/1179559718 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0b65f0 con 0x7f7d3c0b3f90 2024-03-28T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.987+0000 7f7d3b400640 1 --2- 172.21.15.33:0/1179559718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b3f90 0x7f7d3c140fb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0ba5e0 tx=0x7f7d2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e821580c2af49d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.988+0000 7f7d3aa00640 1 -- 172.21.15.33:0/1179559718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28035020 con 0x7f7d3c0b3f90 2024-03-28T01:08:31.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.988+0000 7f7d3aa00640 1 -- 172.21.15.33:0/1179559718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28031830 con 0x7f7d3c0b3f90 2024-03-28T01:08:31.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.988+0000 7f7d3aa00640 1 -- 172.21.15.33:0/1179559718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28031c00 con 0x7f7d3c0b3f90 2024-03-28T01:08:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.988+0000 7f7d41800640 1 -- 172.21.15.33:0/1179559718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b3f90 msgr2=0x7f7d3c140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.988+0000 7f7d41800640 1 --2- 172.21.15.33:0/1179559718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b3f90 0x7f7d3c140fb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0ba5e0 tx=0x7f7d2802b8d0 comp rx=0 tx=0).stop 2024-03-28T01:08:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.989+0000 7f7d41800640 1 -- 172.21.15.33:0/1179559718 shutdown_connections 2024-03-28T01:08:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.989+0000 7f7d41800640 1 --2- 172.21.15.33:0/1179559718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b3f90 0x7f7d3c140fb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.989+0000 7f7d41800640 1 --2- 172.21.15.33:0/1179559718 >> v2:172.21.15.194:3300/0 conn(0x7f7d3c0b53d0 0x7f7d3c14a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.989+0000 7f7d41800640 1 -- 172.21.15.33:0/1179559718 >> 172.21.15.33:0/1179559718 conn(0x7f7d3c0bcf50 msgr2=0x7f7d3c0bd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.989+0000 7f7d41800640 1 -- 172.21.15.33:0/1179559718 shutdown_connections 2024-03-28T01:08:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.989+0000 7f7d41800640 1 -- 172.21.15.33:0/1179559718 wait complete. 2024-03-28T01:08:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.991+0000 7f7d41800640 1 Processor -- start 2024-03-28T01:08:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.991+0000 7f7d41800640 1 -- start start 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d41800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d3c0b3f90 0x7f7d3c1dc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d41800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b53d0 0x7f7d3c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d41800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d3c14ad00 0x7f7d3c0a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d41800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b5930 con 0x7f7d3c0b53d0 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d41800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b57b0 con 0x7f7d3c14ad00 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d41800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b6d60 con 0x7f7d3c0b3f90 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d3b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b53d0 0x7f7d3c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d3b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b53d0 0x7f7d3c0a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32838/0 (socket says 172.21.15.33:32838) 2024-03-28T01:08:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d3b400640 1 -- 172.21.15.33:0/3078616886 learned_addr learned my addr 172.21.15.33:0/3078616886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.993+0000 7f7d40e00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d3c14ad00 0x7f7d3c0a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d3b400640 1 -- 172.21.15.33:0/3078616886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d3c0b3f90 msgr2=0x7f7d3c1dc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d3b400640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d3c0b3f90 0x7f7d3c1dc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d3b400640 1 -- 172.21.15.33:0/3078616886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d3c14ad00 msgr2=0x7f7d3c0a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d3b400640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d3c14ad00 0x7f7d3c0a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d3b400640 1 -- 172.21.15.33:0/3078616886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c1dcfb0 con 0x7f7d3c0b53d0 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d40e00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d3c14ad00 0x7f7d3c0a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d3b400640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b53d0 0x7f7d3c0a98e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000c00 tx=0x7f7d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28040070 con 0x7f7d3c0b53d0 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2803a910 con 0x7f7d3c0b53d0 2024-03-28T01:08:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c0ac470 con 0x7f7d3c0b53d0 2024-03-28T01:08:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.994+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28035040 con 0x7f7d3c0b53d0 2024-03-28T01:08:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.995+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c0ac190 con 0x7f7d3c0b53d0 2024-03-28T01:08:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.996+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d3c075310 con 0x7f7d3c0b53d0 2024-03-28T01:08:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.997+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7d2804b060 con 0x7f7d3c0b53d0 2024-03-28T01:08:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.997+0000 7f7d33e00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7d1c06bbd0 0x7f7d1c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.998+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f7d280ac2f0 con 0x7f7d3c0b53d0 2024-03-28T01:08:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.998+0000 7f7d3be00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7d1c06bbd0 0x7f7d1c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.998+0000 7f7d33e00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7f7d1c074090 0x7f7d1c076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.998+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d2802fe50 con 0x7f7d1c074090 2024-03-28T01:08:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.998+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d280b3050 con 0x7f7d3c0b53d0 2024-03-28T01:08:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.998+0000 7f7d40e00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7f7d1c074090 0x7f7d1c076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.999+0000 7f7d3be00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7d1c06bbd0 0x7f7d1c06e090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006a90 tx=0x7f7d2c006a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:31.999+0000 7f7d40e00640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7f7d1c074090 0x7f7d1c076530 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.000+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== osd.0 v2:172.21.15.33:6817/1178911672 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7d2802fe50 con 0x7f7d1c074090 2024-03-28T01:08:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.029+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 --> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7d3c0bbfb0 con 0x7f7d1c074090 2024-03-28T01:08:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.030+0000 7f7d33e00640 1 -- 172.21.15.33:0/3078616886 <== osd.0 v2:172.21.15.33:6817/1178911672 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7d3c0bbfb0 con 0x7f7d1c074090 2024-03-28T01:08:32.033 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:32.033 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.030+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7f7d1c074090 msgr2=0x7f7d1c076530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.030+0000 7f7d41800640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6817/1178911672,v1:172.21.15.33:6820/1178911672] conn(0x7f7d1c074090 0x7f7d1c076530 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7d1c06bbd0 msgr2=0x7f7d1c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7d1c06bbd0 0x7f7d1c06e090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006a90 tx=0x7f7d2c006a20 comp rx=0 tx=0).stop 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b53d0 msgr2=0x7f7d3c0a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 --2- 172.21.15.33:0/3078616886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c0b53d0 0x7f7d3c0a98e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000c00 tx=0x7f7d280046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d40e00640 1 -- 172.21.15.33:0/3078616886 reap_dead start 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 shutdown_connections 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 >> 172.21.15.33:0/3078616886 conn(0x7f7d3c0bcf50 msgr2=0x7f7d3c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 shutdown_connections 2024-03-28T01:08:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.031+0000 7f7d41800640 1 -- 172.21.15.33:0/3078616886 wait complete. 2024-03-28T01:08:32.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-28T01:08:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.269+0000 7f3c6b200640 1 Processor -- start 2024-03-28T01:08:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.269+0000 7f3c6b200640 1 -- start start 2024-03-28T01:08:32.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.271+0000 7f3c6b200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c6c0b8d00 0x7f3c6c14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.271+0000 7f3c6b200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c0b55b0 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.271+0000 7f3c6b200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c0b9190 con 0x7f3c6c0b51d0 2024-03-28T01:08:32.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.271+0000 7f3c6b200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6c14a820 con 0x7f3c6c0b8d00 2024-03-28T01:08:32.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.271+0000 7f3c69400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c6c0b8d00 0x7f3c6c14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.271+0000 7f3c6a800640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c6c0b6b80 0x7f3c6c14b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43652/0 (socket says 172.21.15.33:43652) 2024-03-28T01:08:32.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.271+0000 7f3c6a800640 1 -- 172.21.15.33:0/2941913722 learned_addr learned my addr 172.21.15.33:0/2941913722 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:32.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.272+0000 7f3c69400640 1 -- 172.21.15.33:0/2941913722 >> v1:172.21.15.194:6789/0 conn(0x7f3c6c0b51d0 legacy=0x7f3c6c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:32.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.272+0000 7f3c69400640 1 -- 172.21.15.33:0/2941913722 >> v1:172.21.15.33:6789/0 conn(0x7f3c6c0b6b80 legacy=0x7f3c6c14b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.272+0000 7f3c69400640 1 -- 172.21.15.33:0/2941913722 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c0b9800 con 0x7f3c6c0b8d00 2024-03-28T01:08:32.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.272+0000 7f3c68a00640 1 -- 172.21.15.33:0/2941913722 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474747756 0 0) 0x7f3c6c0b55b0 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.273+0000 7f3c69400640 1 --2- 172.21.15.33:0/2941913722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b8d00 0x7f3c6c14ac70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c580097a0 tx=0x7f3c5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730cc936ccab0d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c68a00640 1 -- 172.21.15.33:0/2941913722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58035020 con 0x7f3c6c0b8d00 2024-03-28T01:08:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c68a00640 1 -- 172.21.15.33:0/2941913722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58031830 con 0x7f3c6c0b8d00 2024-03-28T01:08:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c68a00640 1 -- 172.21.15.33:0/2941913722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58031c00 con 0x7f3c6c0b8d00 2024-03-28T01:08:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c6b200640 1 -- 172.21.15.33:0/2941913722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b8d00 msgr2=0x7f3c6c14ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c6b200640 1 --2- 172.21.15.33:0/2941913722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b8d00 0x7f3c6c14ac70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c580097a0 tx=0x7f3c5802b8d0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c6b200640 1 -- 172.21.15.33:0/2941913722 shutdown_connections 2024-03-28T01:08:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c6b200640 1 --2- 172.21.15.33:0/2941913722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b8d00 0x7f3c6c14ac70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c6b200640 1 -- 172.21.15.33:0/2941913722 >> 172.21.15.33:0/2941913722 conn(0x7f3c6c072940 msgr2=0x7f3c6c072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.274+0000 7f3c6b200640 1 -- 172.21.15.33:0/2941913722 shutdown_connections 2024-03-28T01:08:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.275+0000 7f3c6b200640 1 -- 172.21.15.33:0/2941913722 wait complete. 2024-03-28T01:08:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.276+0000 7f3c6b200640 1 Processor -- start 2024-03-28T01:08:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.276+0000 7f3c6b200640 1 -- start start 2024-03-28T01:08:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c6b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6c0b51d0 0x7f3c6c1471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c6b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b6b80 0x7f3c6c1dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c6b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3c6c144720 0x7f3c6c1dd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c6b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0b39c0 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c6b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0b44c0 con 0x7f3c6c0b51d0 2024-03-28T01:08:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c6b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c152b80 con 0x7f3c6c144720 2024-03-28T01:08:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c69400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b6b80 0x7f3c6c1dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c69400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b6b80 0x7f3c6c1dd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32854/0 (socket says 172.21.15.33:32854) 2024-03-28T01:08:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.277+0000 7f3c69400640 1 -- 172.21.15.33:0/908568199 learned_addr learned my addr 172.21.15.33:0/908568199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69e00640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6c0b51d0 0x7f3c6c1471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69400640 1 -- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3c6c144720 msgr2=0x7f3c6c1dd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c6a800640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3c6c144720 0x7f3c6c1dd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69400640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3c6c144720 0x7f3c6c1dd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69400640 1 -- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6c0b51d0 msgr2=0x7f3c6c1471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69400640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6c0b51d0 0x7f3c6c1471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69400640 1 -- 172.21.15.33:0/908568199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c1dcd00 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69e00640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6c0b51d0 0x7f3c6c1471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c6a800640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3c6c144720 0x7f3c6c1dd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c69400640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b6b80 0x7f3c6c1dd1e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c580097a0 tx=0x7f3c580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c62000640 1 -- 172.21.15.33:0/908568199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58004230 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c62000640 1 -- 172.21.15.33:0/908568199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c580043f0 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c62000640 1 -- 172.21.15.33:0/908568199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58003850 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.278+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c13ddd0 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.279+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c13daf0 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.280+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c1dced0 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.282+0000 7f3c62000640 1 -- 172.21.15.33:0/908568199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3c58038020 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.282+0000 7f3c62000640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3c4c06bbd0 0x7f3c4c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.283+0000 7f3c62000640 1 -- 172.21.15.33:0/908568199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f3c580ac630 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.283+0000 7f3c69e00640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3c4c06bbd0 0x7f3c4c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.283+0000 7f3c69e00640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3c4c06bbd0 0x7f3c4c06e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c14b5b0 tx=0x7f3c5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.292+0000 7f3c62000640 1 -- 172.21.15.33:0/908568199 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3c58040070 con 0x7f3c6c0b6b80 2024-03-28T01:08:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.498+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c6c075310 con 0x7f3c4c06bbd0 2024-03-28T01:08:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.499+0000 7f3c62000640 1 -- 172.21.15.33:0/908568199 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3c6c075310 con 0x7f3c4c06bbd0 2024-03-28T01:08:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 0 to deep-scrub 2024-03-28T01:08:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3c4c06bbd0 msgr2=0x7f3c4c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3c4c06bbd0 0x7f3c4c06e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c14b5b0 tx=0x7f3c5c009040 comp rx=0 tx=0).stop 2024-03-28T01:08:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b6b80 msgr2=0x7f3c6c1dd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b6b80 0x7f3c6c1dd1e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c580097a0 tx=0x7f3c580046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 shutdown_connections 2024-03-28T01:08:32.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3c4c06bbd0 0x7f3c4c06e090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3c6c144720 0x7f3c6c1dd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c6c0b6b80 0x7f3c6c1dd1e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 --2- 172.21.15.33:0/908568199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6c0b51d0 0x7f3c6c1471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 >> 172.21.15.33:0/908568199 conn(0x7f3c6c072940 msgr2=0x7f3c6c0bcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 shutdown_connections 2024-03-28T01:08:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.502+0000 7f3c6b200640 1 -- 172.21.15.33:0/908568199 wait complete. 2024-03-28T01:08:32.522 INFO:tasks.ceph:Scrubbing osd.1 2024-03-28T01:08:32.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.731+0000 7f643a800640 1 Processor -- start 2024-03-28T01:08:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.731+0000 7f643a800640 1 -- start start 2024-03-28T01:08:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.732+0000 7f643a800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6434140f30 0x7f6434141390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.732+0000 7f643a800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340bd470 con 0x7f64340bd090 2024-03-28T01:08:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.732+0000 7f643a800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340b3f90 con 0x7f643414fa80 2024-03-28T01:08:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.732+0000 7f643a800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64340b6630 con 0x7f6434140f30 2024-03-28T01:08:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6439400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f64340bd090 0x7f643414a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58382/0 (socket says 172.21.15.33:58382) 2024-03-28T01:08:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6439400640 1 -- 172.21.15.33:0/2105230760 learned_addr learned my addr 172.21.15.33:0/2105230760 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6438a00640 1 --2- 172.21.15.33:0/2105230760 >> v2:172.21.15.33:3300/0 conn(0x7f6434140f30 0x7f6434141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6433e00640 1 -- 172.21.15.33:0/2105230760 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825099555 0 0) 0x7f64340bd470 con 0x7f64340bd090 2024-03-28T01:08:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6433e00640 1 -- 172.21.15.33:0/2105230760 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f641c003520 con 0x7f64340bd090 2024-03-28T01:08:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6438a00640 1 -- 172.21.15.33:0/2105230760 >> v1:172.21.15.194:6789/0 conn(0x7f643414fa80 legacy=0x7f6434151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6438a00640 1 -- 172.21.15.33:0/2105230760 >> v1:172.21.15.33:6789/0 conn(0x7f64340bd090 legacy=0x7f643414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.733+0000 7f6438a00640 1 -- 172.21.15.33:0/2105230760 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340b4450 con 0x7f6434140f30 2024-03-28T01:08:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.735+0000 7f6438a00640 1 --2- 172.21.15.33:0/2105230760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6434140f30 0x7f6434141390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64200097a0 tx=0x7f642002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cdb68d91e3929b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.735+0000 7f6433e00640 1 -- 172.21.15.33:0/2105230760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420034070 con 0x7f6434140f30 2024-03-28T01:08:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.735+0000 7f6433e00640 1 -- 172.21.15.33:0/2105230760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6420030610 con 0x7f6434140f30 2024-03-28T01:08:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.735+0000 7f6433e00640 1 -- 172.21.15.33:0/2105230760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64200309e0 con 0x7f6434140f30 2024-03-28T01:08:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.735+0000 7f643a800640 1 -- 172.21.15.33:0/2105230760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6434140f30 msgr2=0x7f6434141390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.735+0000 7f643a800640 1 --2- 172.21.15.33:0/2105230760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6434140f30 0x7f6434141390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64200097a0 tx=0x7f642002b960 comp rx=0 tx=0).stop 2024-03-28T01:08:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.736+0000 7f643a800640 1 -- 172.21.15.33:0/2105230760 shutdown_connections 2024-03-28T01:08:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.736+0000 7f643a800640 1 --2- 172.21.15.33:0/2105230760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6434140f30 0x7f6434141390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.736+0000 7f643a800640 1 -- 172.21.15.33:0/2105230760 >> 172.21.15.33:0/2105230760 conn(0x7f6434072940 msgr2=0x7f6434072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.736+0000 7f643a800640 1 -- 172.21.15.33:0/2105230760 shutdown_connections 2024-03-28T01:08:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.736+0000 7f643a800640 1 -- 172.21.15.33:0/2105230760 wait complete. 2024-03-28T01:08:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.737+0000 7f643a800640 1 Processor -- start 2024-03-28T01:08:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.737+0000 7f643a800640 1 -- start start 2024-03-28T01:08:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f643a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64340bd090 0x7f6434137820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f643a800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6434140f30 0x7f6434137d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f643a800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f643414fa80 0x7f64341dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f643a800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340b4730 con 0x7f64340bd090 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f643a800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340b5400 con 0x7f6434140f30 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f643a800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340b6290 con 0x7f643414fa80 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f6439400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64340bd090 0x7f6434137820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f6439400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64340bd090 0x7f6434137820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59270/0 (socket says 172.21.15.33:59270) 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.738+0000 7f6439400640 1 -- 172.21.15.33:0/4004763343 learned_addr learned my addr 172.21.15.33:0/4004763343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6438a00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6434140f30 0x7f6434137d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439e00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f643414fa80 0x7f64341dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439400640 1 -- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f643414fa80 msgr2=0x7f64341dcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439400640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f643414fa80 0x7f64341dcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439400640 1 -- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6434140f30 msgr2=0x7f6434137d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439400640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6434140f30 0x7f6434137d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439400640 1 -- 172.21.15.33:0/4004763343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64341438c0 con 0x7f64340bd090 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6438a00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6434140f30 0x7f6434137d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439e00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f643414fa80 0x7f64341dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6439400640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64340bd090 0x7f6434137820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f642800c410 tx=0x7f642800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428017070 con 0x7f64340bd090 2024-03-28T01:08:32.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6428005ce0 con 0x7f64340bd090 2024-03-28T01:08:32.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280072f0 con 0x7f64340bd090 2024-03-28T01:08:32.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.739+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643414f760 con 0x7f64340bd090 2024-03-28T01:08:32.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.740+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64341382a0 con 0x7f64340bd090 2024-03-28T01:08:32.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.741+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6434075310 con 0x7f64340bd090 2024-03-28T01:08:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.743+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f642801a070 con 0x7f64340bd090 2024-03-28T01:08:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.744+0000 7f6431600640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f641406bbd0 0x7f641406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.744+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f6428089ff0 con 0x7f64340bd090 2024-03-28T01:08:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.744+0000 7f6438a00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f641406bbd0 0x7f641406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.744+0000 7f6431600640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7f6414074080 0x7f6414076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.744+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 --> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6428015e70 con 0x7f6414074080 2024-03-28T01:08:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.744+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f642808c5d0 con 0x7f64340bd090 2024-03-28T01:08:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.744+0000 7f6439e00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7f6414074080 0x7f6414076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.745+0000 7f6438a00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f641406bbd0 0x7f641406e090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6420036000 tx=0x7f642003b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.745+0000 7f6439e00640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7f6414074080 0x7f6414076520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.746+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== osd.1 v2:172.21.15.33:6816/4011280304 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6428015e70 con 0x7f6414074080 2024-03-28T01:08:32.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.774+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 --> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f64340b6070 con 0x7f6414074080 2024-03-28T01:08:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.774+0000 7f6431600640 1 -- 172.21.15.33:0/4004763343 <== osd.1 v2:172.21.15.33:6816/4011280304 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f64340b6070 con 0x7f6414074080 2024-03-28T01:08:32.777 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:32.778 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7f6414074080 msgr2=0x7f6414076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6816/4011280304,v1:172.21.15.33:6818/4011280304] conn(0x7f6414074080 0x7f6414076520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f641406bbd0 msgr2=0x7f641406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f641406bbd0 0x7f641406e090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6420036000 tx=0x7f642003b040 comp rx=0 tx=0).stop 2024-03-28T01:08:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64340bd090 msgr2=0x7f6434137820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 --2- 172.21.15.33:0/4004763343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64340bd090 0x7f6434137820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f642800c410 tx=0x7f642800c8d0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f6439e00640 1 -- 172.21.15.33:0/4004763343 reap_dead start 2024-03-28T01:08:32.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 shutdown_connections 2024-03-28T01:08:32.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 >> 172.21.15.33:0/4004763343 conn(0x7f6434072940 msgr2=0x7f64340b9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:32.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 shutdown_connections 2024-03-28T01:08:32.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.775+0000 7f643a800640 1 -- 172.21.15.33:0/4004763343 wait complete. 2024-03-28T01:08:32.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-28T01:08:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.991+0000 7f8263e00640 1 Processor -- start 2024-03-28T01:08:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.991+0000 7f8263e00640 1 -- start start 2024-03-28T01:08:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.992+0000 7f8263e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82640b6f60 0x7f826413ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.992+0000 7f8263e00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f82641485b0 0x7f826413f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.992+0000 7f8263e00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826413f710 con 0x7f82640b7b60 2024-03-28T01:08:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.992+0000 7f8263e00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640b7f40 con 0x7f82641485b0 2024-03-28T01:08:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.992+0000 7f8263e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b23a0 con 0x7f82640b6f60 2024-03-28T01:08:32.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.993+0000 7f825be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82640b6f60 0x7f826413ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.993+0000 7f8263400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f82641485b0 0x7f826413f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.993+0000 7f8263400640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f82641485b0 0x7f826413f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:35756/0 (socket says 172.21.15.33:35756) 2024-03-28T01:08:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.993+0000 7f8263400640 1 -- 172.21.15.33:0/3972963812 learned_addr learned my addr 172.21.15.33:0/3972963812 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.993+0000 7f825be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82640b6f60 0x7f826413ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.33:49850/0 (socket says 172.21.15.33:49850) 2024-03-28T01:08:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.994+0000 7f825be00640 1 -- 172.21.15.33:0/3972963812 >> v2:172.21.15.194:3300/0 conn(0x7f82641485b0 msgr2=0x7f826413f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.994+0000 7f825be00640 1 --2- 172.21.15.33:0/3972963812 >> v2:172.21.15.194:3300/0 conn(0x7f82641485b0 0x7f826413f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.994+0000 7f825be00640 1 -- 172.21.15.33:0/3972963812 >> v1:172.21.15.194:6789/0 conn(0x7f82640b7b60 legacy=0x7f8264072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.994+0000 7f825be00640 1 -- 172.21.15.33:0/3972963812 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264071bb0 con 0x7f82640b6f60 2024-03-28T01:08:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.995+0000 7f825be00640 1 --2- 172.21.15.33:0/3972963812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82640b6f60 0x7f826413ec90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f824c0097a0 tx=0x7f824c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d23bba3755ab05e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8262000640 1 -- 172.21.15.33:0/3972963812 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c0315a0 con 0x7f82640b6f60 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8262000640 1 -- 172.21.15.33:0/3972963812 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c037070 con 0x7f82640b6f60 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8262000640 1 -- 172.21.15.33:0/3972963812 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c031960 con 0x7f82640b6f60 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 -- 172.21.15.33:0/3972963812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82640b6f60 msgr2=0x7f826413ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 --2- 172.21.15.33:0/3972963812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82640b6f60 0x7f826413ec90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f824c0097a0 tx=0x7f824c02b840 comp rx=0 tx=0).stop 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 -- 172.21.15.33:0/3972963812 shutdown_connections 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 --2- 172.21.15.33:0/3972963812 >> v2:172.21.15.194:3300/0 conn(0x7f82641485b0 0x7f826413f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 --2- 172.21.15.33:0/3972963812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82640b6f60 0x7f826413ec90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 -- 172.21.15.33:0/3972963812 >> 172.21.15.33:0/3972963812 conn(0x7f826406b450 msgr2=0x7f826406b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 -- 172.21.15.33:0/3972963812 shutdown_connections 2024-03-28T01:08:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.996+0000 7f8263e00640 1 -- 172.21.15.33:0/3972963812 wait complete. 2024-03-28T01:08:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.998+0000 7f8263e00640 1 Processor -- start 2024-03-28T01:08:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.998+0000 7f8263e00640 1 -- start start 2024-03-28T01:08:33.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f8263e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82640b6f60 0x7f826406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f8263e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82640b7b60 0x7f8264147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f8263e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82641485b0 0x7f82641399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f8263e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b4ca0 con 0x7f82640b7b60 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f8263e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b2000 con 0x7f82640b6f60 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f8263e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b4e20 con 0x7f82641485b0 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f825be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82640b7b60 0x7f8264147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f825be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82640b7b60 0x7f8264147ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59284/0 (socket says 172.21.15.33:59284) 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f825be00640 1 -- 172.21.15.33:0/4179497455 learned_addr learned my addr 172.21.15.33:0/4179497455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:32.999+0000 7f8262a00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82640b6f60 0x7f826406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f8263400640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82641485b0 0x7f82641399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f825be00640 1 -- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82641485b0 msgr2=0x7f82641399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f825be00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82641485b0 0x7f82641399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f825be00640 1 -- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82640b6f60 msgr2=0x7f826406d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f825be00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82640b6f60 0x7f826406d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f825be00640 1 -- 172.21.15.33:0/4179497455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264141ab0 con 0x7f82640b7b60 2024-03-28T01:08:33.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f8263400640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82641485b0 0x7f82641399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:08:33.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.000+0000 7f825be00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82640b7b60 0x7f8264147ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f824c034040 tx=0x7f824c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.001+0000 7f825aa00640 1 -- 172.21.15.33:0/4179497455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c032630 con 0x7f82640b7b60 2024-03-28T01:08:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.001+0000 7f825aa00640 1 -- 172.21.15.33:0/4179497455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c0327f0 con 0x7f82640b7b60 2024-03-28T01:08:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.001+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8264142350 con 0x7f82640b7b60 2024-03-28T01:08:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.001+0000 7f825aa00640 1 -- 172.21.15.33:0/4179497455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c032be0 con 0x7f82640b7b60 2024-03-28T01:08:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.001+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826406def0 con 0x7f82640b7b60 2024-03-28T01:08:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.002+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8264075310 con 0x7f82640b7b60 2024-03-28T01:08:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.003+0000 7f825aa00640 1 -- 172.21.15.33:0/4179497455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f824c03b3f0 con 0x7f82640b7b60 2024-03-28T01:08:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.004+0000 7f825aa00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f824006bc00 0x7f824006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.005+0000 7f825aa00640 1 -- 172.21.15.33:0/4179497455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f824c02f030 con 0x7f82640b7b60 2024-03-28T01:08:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.005+0000 7f8262a00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f824006bc00 0x7f824006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.005+0000 7f8262a00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f824006bc00 0x7f824006e0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8254000f60 tx=0x7f8254009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.008+0000 7f825aa00640 1 -- 172.21.15.33:0/4179497455 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f824c040070 con 0x7f82640b7b60 2024-03-28T01:08:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.245+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f82641450a0 con 0x7f824006bc00 2024-03-28T01:08:33.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.246+0000 7f825aa00640 1 -- 172.21.15.33:0/4179497455 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f82641450a0 con 0x7f824006bc00 2024-03-28T01:08:33.248 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 1 to deep-scrub 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f824006bc00 msgr2=0x7f824006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f824006bc00 0x7f824006e0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8254000f60 tx=0x7f8254009040 comp rx=0 tx=0).stop 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82640b7b60 msgr2=0x7f8264147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82640b7b60 0x7f8264147ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f824c034040 tx=0x7f824c0046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 shutdown_connections 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f824006bc00 0x7f824006e0c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82641485b0 0x7f82641399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82640b7b60 0x7f8264147ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 --2- 172.21.15.33:0/4179497455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82640b6f60 0x7f826406d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 >> 172.21.15.33:0/4179497455 conn(0x7f826406b450 msgr2=0x7f8264145930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 shutdown_connections 2024-03-28T01:08:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.249+0000 7f8263e00640 1 -- 172.21.15.33:0/4179497455 wait complete. 2024-03-28T01:08:33.271 INFO:tasks.ceph:Scrubbing osd.2 2024-03-28T01:08:33.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.487+0000 7fc0c9800640 1 Processor -- start 2024-03-28T01:08:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.487+0000 7fc0c9800640 1 -- start start 2024-03-28T01:08:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.488+0000 7fc0c9800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fc0c409df20 0x7fc0c40b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.488+0000 7fc0c9800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0c414d960 0x7fc0c414fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c9800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c40b3cd0 con 0x7fc0c40affd0 2024-03-28T01:08:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c9800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c413f930 con 0x7fc0c414d960 2024-03-28T01:08:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c9800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c40b17e0 con 0x7fc0c409df20 2024-03-28T01:08:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c3400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0c414d960 0x7fc0c414fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c2000640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc0c40affd0 0x7fc0c4145870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58396/0 (socket says 172.21.15.33:58396) 2024-03-28T01:08:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c2000640 1 -- 172.21.15.33:0/2374068932 learned_addr learned my addr 172.21.15.33:0/2374068932 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c2a00640 1 --2- 172.21.15.33:0/2374068932 >> v2:172.21.15.194:3300/0 conn(0x7fc0c409df20 0x7fc0c40b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c3400640 1 -- 172.21.15.33:0/2374068932 >> v2:172.21.15.194:3300/0 conn(0x7fc0c409df20 msgr2=0x7fc0c40b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c3400640 1 --2- 172.21.15.33:0/2374068932 >> v2:172.21.15.194:3300/0 conn(0x7fc0c409df20 0x7fc0c40b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.489+0000 7fc0c3400640 1 -- 172.21.15.33:0/2374068932 >> v1:172.21.15.33:6789/0 conn(0x7fc0c40affd0 legacy=0x7fc0c4145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.490+0000 7fc0c3400640 1 -- 172.21.15.33:0/2374068932 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c41450d0 con 0x7fc0c414d960 2024-03-28T01:08:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.490+0000 7fc0c1600640 1 -- 172.21.15.33:0/2374068932 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428889546 0 0) 0x7fc0b0001110 con 0x7fc0c40affd0 2024-03-28T01:08:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.491+0000 7fc0c3400640 1 --2- 172.21.15.33:0/2374068932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c414fd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40afcb0 tx=0x7fc0bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffbb6228cf6f194 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.491+0000 7fc0c1600640 1 -- 172.21.15.33:0/2374068932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc039070 con 0x7fc0c414d960 2024-03-28T01:08:33.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.491+0000 7fc0c1600640 1 -- 172.21.15.33:0/2374068932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc02cc60 con 0x7fc0c414d960 2024-03-28T01:08:33.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.491+0000 7fc0c1600640 1 -- 172.21.15.33:0/2374068932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc034660 con 0x7fc0c414d960 2024-03-28T01:08:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 -- 172.21.15.33:0/2374068932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 msgr2=0x7fc0c414fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 --2- 172.21.15.33:0/2374068932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c414fd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40afcb0 tx=0x7fc0bc02beb0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 -- 172.21.15.33:0/2374068932 shutdown_connections 2024-03-28T01:08:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 --2- 172.21.15.33:0/2374068932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c414fd50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 --2- 172.21.15.33:0/2374068932 >> v2:172.21.15.194:3300/0 conn(0x7fc0c409df20 0x7fc0c40b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 -- 172.21.15.33:0/2374068932 >> 172.21.15.33:0/2374068932 conn(0x7fc0c406b210 msgr2=0x7fc0c406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 -- 172.21.15.33:0/2374068932 shutdown_connections 2024-03-28T01:08:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.492+0000 7fc0c9800640 1 -- 172.21.15.33:0/2374068932 wait complete. 2024-03-28T01:08:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.493+0000 7fc0c9800640 1 Processor -- start 2024-03-28T01:08:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.494+0000 7fc0c9800640 1 -- start start 2024-03-28T01:08:33.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.494+0000 7fc0c9800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0c409df20 0x7fc0c409d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.494+0000 7fc0c9800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c40affd0 0x7fc0c413b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.494+0000 7fc0c9800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c413b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.494+0000 7fc0c9800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c4145350 con 0x7fc0c414d960 2024-03-28T01:08:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.494+0000 7fc0c9800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c409e300 con 0x7fc0c40affd0 2024-03-28T01:08:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.494+0000 7fc0c9800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40b21b0 con 0x7fc0c409df20 2024-03-28T01:08:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c413b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c413b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59300/0 (socket says 172.21.15.33:59300) 2024-03-28T01:08:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 -- 172.21.15.33:0/1596539200 learned_addr learned my addr 172.21.15.33:0/1596539200 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c2a00640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0c409df20 0x7fc0c409d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c2000640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c40affd0 0x7fc0c413b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 -- 172.21.15.33:0/1596539200 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0c409df20 msgr2=0x7fc0c409d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0c409df20 0x7fc0c409d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 -- 172.21.15.33:0/1596539200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c40affd0 msgr2=0x7fc0c413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c40affd0 0x7fc0c413b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.495+0000 7fc0c3400640 1 -- 172.21.15.33:0/1596539200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c413ca90 con 0x7fc0c414d960 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.496+0000 7fc0c3400640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c413b650 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009d90 tx=0x7fc0bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.496+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc039050 con 0x7fc0c414d960 2024-03-28T01:08:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.496+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc045070 con 0x7fc0c414d960 2024-03-28T01:08:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.496+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c413cd70 con 0x7fc0c414d960 2024-03-28T01:08:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.496+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc03e830 con 0x7fc0c414d960 2024-03-28T01:08:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.496+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c414d6c0 con 0x7fc0c414d960 2024-03-28T01:08:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.497+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0c409c500 con 0x7fc0c414d960 2024-03-28T01:08:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.498+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc0bc03e9d0 con 0x7fc0c414d960 2024-03-28T01:08:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.499+0000 7fc0baa00640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc0a406bc20 0x7fc0a406e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.499+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fc0bc0ad950 con 0x7fc0c414d960 2024-03-28T01:08:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.499+0000 7fc0c2a00640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc0a406bc20 0x7fc0a406e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.499+0000 7fc0baa00640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7fc0a4074160 0x7fc0a4076600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.500+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0a4076cd0 con 0x7fc0a4074160 2024-03-28T01:08:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.500+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0bc0b22a0 con 0x7fc0c414d960 2024-03-28T01:08:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.500+0000 7fc0c2000640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7fc0a4074160 0x7fc0a4076600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.500+0000 7fc0c2a00640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc0a406bc20 0x7fc0a406e0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4006fd0 tx=0x7fc0b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.500+0000 7fc0c2000640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7fc0a4074160 0x7fc0a4076600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.501+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== osd.2 v2:172.21.15.33:6808/1101919847 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc0a4076cd0 con 0x7fc0a4074160 2024-03-28T01:08:33.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.516+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 --> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc0c4075380 con 0x7fc0a4074160 2024-03-28T01:08:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.516+0000 7fc0baa00640 1 -- 172.21.15.33:0/1596539200 <== osd.2 v2:172.21.15.33:6808/1101919847 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc0c4075380 con 0x7fc0a4074160 2024-03-28T01:08:33.519 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:33.519 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:33.519 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7fc0a4074160 msgr2=0x7fc0a4076600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6808/1101919847,v1:172.21.15.33:6809/1101919847] conn(0x7fc0a4074160 0x7fc0a4076600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc0a406bc20 msgr2=0x7fc0a406e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc0a406bc20 0x7fc0a406e0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4006fd0 tx=0x7fc0b4008040 comp rx=0 tx=0).stop 2024-03-28T01:08:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 msgr2=0x7fc0c413b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 --2- 172.21.15.33:0/1596539200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c414d960 0x7fc0c413b650 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009d90 tx=0x7fc0bc004a40 comp rx=0 tx=0).stop 2024-03-28T01:08:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c3400640 1 -- 172.21.15.33:0/1596539200 reap_dead start 2024-03-28T01:08:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 shutdown_connections 2024-03-28T01:08:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 >> 172.21.15.33:0/1596539200 conn(0x7fc0c406b210 msgr2=0x7fc0c406c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 shutdown_connections 2024-03-28T01:08:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.517+0000 7fc0c9800640 1 -- 172.21.15.33:0/1596539200 wait complete. 2024-03-28T01:08:33.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-28T01:08:33.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.761+0000 7f46fbe00640 1 Processor -- start 2024-03-28T01:08:33.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.761+0000 7f46fbe00640 1 -- start start 2024-03-28T01:08:33.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.762+0000 7f46fbe00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46fc0bd050 0x7f46fc14a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.762+0000 7f46fbe00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f46fc141510 0x7f46fc151a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.762+0000 7f46fbe00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc0bd430 con 0x7f46fc14ac20 2024-03-28T01:08:33.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.762+0000 7f46fbe00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc0b6370 con 0x7f46fc0bd050 2024-03-28T01:08:33.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.762+0000 7f46fbe00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc0b3f90 con 0x7f46fc141510 2024-03-28T01:08:33.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46faa00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46fc0bd050 0x7f46fc14a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46faa00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46fc0bd050 0x7f46fc14a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59310/0 (socket says 172.21.15.33:59310) 2024-03-28T01:08:33.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46faa00640 1 -- 172.21.15.33:0/2732322853 learned_addr learned my addr 172.21.15.33:0/2732322853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:33.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46fb400640 1 --2- 172.21.15.33:0/2732322853 >> v2:172.21.15.194:3300/0 conn(0x7f46fc141510 0x7f46fc151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46faa00640 1 -- 172.21.15.33:0/2732322853 >> v2:172.21.15.194:3300/0 conn(0x7f46fc141510 msgr2=0x7f46fc151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:33.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46faa00640 1 --2- 172.21.15.33:0/2732322853 >> v2:172.21.15.194:3300/0 conn(0x7f46fc141510 0x7f46fc151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46faa00640 1 -- 172.21.15.33:0/2732322853 >> v1:172.21.15.194:6789/0 conn(0x7f46fc14ac20 legacy=0x7f46fc140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:33.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.763+0000 7f46faa00640 1 -- 172.21.15.33:0/2732322853 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc0b58a0 con 0x7f46fc0bd050 2024-03-28T01:08:33.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.764+0000 7f46faa00640 1 --2- 172.21.15.33:0/2732322853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc14a6e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0bdb60 tx=0x7f46ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4896307dd63b8dd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.765+0000 7f46f9600640 1 -- 172.21.15.33:0/2732322853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec02e070 con 0x7f46fc0bd050 2024-03-28T01:08:33.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.765+0000 7f46f9600640 1 -- 172.21.15.33:0/2732322853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec037070 con 0x7f46fc0bd050 2024-03-28T01:08:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.765+0000 7f46f9600640 1 -- 172.21.15.33:0/2732322853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec031a00 con 0x7f46fc0bd050 2024-03-28T01:08:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.765+0000 7f46fbe00640 1 -- 172.21.15.33:0/2732322853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 msgr2=0x7f46fc14a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.765+0000 7f46fbe00640 1 --2- 172.21.15.33:0/2732322853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc14a6e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0bdb60 tx=0x7f46ec02b880 comp rx=0 tx=0).stop 2024-03-28T01:08:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.766+0000 7f46fbe00640 1 -- 172.21.15.33:0/2732322853 shutdown_connections 2024-03-28T01:08:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.766+0000 7f46fbe00640 1 --2- 172.21.15.33:0/2732322853 >> v2:172.21.15.194:3300/0 conn(0x7f46fc141510 0x7f46fc151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.766+0000 7f46fbe00640 1 --2- 172.21.15.33:0/2732322853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc14a6e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.766+0000 7f46fbe00640 1 -- 172.21.15.33:0/2732322853 >> 172.21.15.33:0/2732322853 conn(0x7f46fc072940 msgr2=0x7f46fc072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.766+0000 7f46fbe00640 1 -- 172.21.15.33:0/2732322853 shutdown_connections 2024-03-28T01:08:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.766+0000 7f46fbe00640 1 -- 172.21.15.33:0/2732322853 wait complete. 2024-03-28T01:08:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.767+0000 7f46fbe00640 1 Processor -- start 2024-03-28T01:08:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.767+0000 7f46fbe00640 1 -- start start 2024-03-28T01:08:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.768+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc1426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.768+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46fc141510 0x7f46fc1dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.768+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46fc14ac20 0x7f46fc1dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.768+0000 7f46fbe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0b4730 con 0x7f46fc0bd050 2024-03-28T01:08:33.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.768+0000 7f46fbe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0b66f0 con 0x7f46fc14ac20 2024-03-28T01:08:33.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.768+0000 7f46fbe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0b50a0 con 0x7f46fc141510 2024-03-28T01:08:33.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.768+0000 7f46faa00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc1426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc1426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59320/0 (socket says 172.21.15.33:59320) 2024-03-28T01:08:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 -- 172.21.15.33:0/4111020547 learned_addr learned my addr 172.21.15.33:0/4111020547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46fa000640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46fc141510 0x7f46fc1dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 -- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46fc141510 msgr2=0x7f46fc1dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46fb400640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46fc14ac20 0x7f46fc1dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46fc141510 0x7f46fc1dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 -- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46fc14ac20 msgr2=0x7f46fc1dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46fc14ac20 0x7f46fc1dd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 -- 172.21.15.33:0/4111020547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc1419e0 con 0x7f46fc0bd050 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46fa000640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46fc141510 0x7f46fc1dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.769+0000 7f46faa00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc1426f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f46ec000c00 tx=0x7f46ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.770+0000 7f46f2a00640 1 -- 172.21.15.33:0/4111020547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec032880 con 0x7f46fc0bd050 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.770+0000 7f46f2a00640 1 -- 172.21.15.33:0/4111020547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec032a40 con 0x7f46fc0bd050 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.770+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc142e80 con 0x7f46fc0bd050 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.770+0000 7f46f2a00640 1 -- 172.21.15.33:0/4111020547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec032e30 con 0x7f46fc0bd050 2024-03-28T01:08:33.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.770+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc0b35d0 con 0x7f46fc0bd050 2024-03-28T01:08:33.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.771+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc075310 con 0x7f46fc0bd050 2024-03-28T01:08:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.772+0000 7f46f2a00640 1 -- 172.21.15.33:0/4111020547 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f46ec04c060 con 0x7f46fc0bd050 2024-03-28T01:08:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.774+0000 7f46f2a00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f46dc06bbd0 0x7f46dc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.774+0000 7f46f2a00640 1 -- 172.21.15.33:0/4111020547 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f46ec0ac240 con 0x7f46fc0bd050 2024-03-28T01:08:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.774+0000 7f46fa000640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f46dc06bbd0 0x7f46dc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.775+0000 7f46fa000640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f46dc06bbd0 0x7f46dc06e090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f46fc13e240 tx=0x7f46e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:33.778+0000 7f46f2a00640 1 -- 172.21.15.33:0/4111020547 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f46ec040070 con 0x7f46fc0bd050 2024-03-28T01:08:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.047+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f46fc144b10 con 0x7f46dc06bbd0 2024-03-28T01:08:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.047+0000 7f46f2a00640 1 -- 172.21.15.33:0/4111020547 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f46fc144b10 con 0x7f46dc06bbd0 2024-03-28T01:08:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 2 to deep-scrub 2024-03-28T01:08:34.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f46dc06bbd0 msgr2=0x7f46dc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f46dc06bbd0 0x7f46dc06e090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f46fc13e240 tx=0x7f46e8008040 comp rx=0 tx=0).stop 2024-03-28T01:08:34.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 msgr2=0x7f46fc1426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc1426f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f46ec000c00 tx=0x7f46ec0046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 shutdown_connections 2024-03-28T01:08:34.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f46dc06bbd0 0x7f46dc06e090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46fc14ac20 0x7f46fc1dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46fc141510 0x7f46fc1dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 --2- 172.21.15.33:0/4111020547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46fc0bd050 0x7f46fc1426f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 >> 172.21.15.33:0/4111020547 conn(0x7f46fc072940 msgr2=0x7f46fc0b9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:34.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 shutdown_connections 2024-03-28T01:08:34.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.050+0000 7f46fbe00640 1 -- 172.21.15.33:0/4111020547 wait complete. 2024-03-28T01:08:34.074 INFO:tasks.ceph:Scrubbing osd.3 2024-03-28T01:08:34.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:34.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.290+0000 7f2cf6a00640 1 Processor -- start 2024-03-28T01:08:34.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.290+0000 7f2cf6a00640 1 -- start start 2024-03-28T01:08:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf6a00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f2cf00b53d0 0x7f2cf014a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf6a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2cf00b3f90 0x7f2cf0140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf014ad00 0x7f2cf01414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf6a00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf0072ea0 con 0x7f2cf00b3f90 2024-03-28T01:08:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf6a00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf00b4710 con 0x7f2cf00b53d0 2024-03-28T01:08:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0072b30 con 0x7f2cf014ad00 2024-03-28T01:08:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf4c00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2cf00b3f90 0x7f2cf0140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf4c00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2cf00b3f90 0x7f2cf0140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59330/0 (socket says 172.21.15.33:59330) 2024-03-28T01:08:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.292+0000 7f2cf4c00640 1 -- 172.21.15.33:0/2063556442 learned_addr learned my addr 172.21.15.33:0/2063556442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf5600640 1 --2- 172.21.15.33:0/2063556442 >> v2:172.21.15.194:3300/0 conn(0x7f2cf00b53d0 0x7f2cf014a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf6000640 1 --2- 172.21.15.33:0/2063556442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf014ad00 0x7f2cf01414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf4c00640 1 -- 172.21.15.33:0/2063556442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf014ad00 msgr2=0x7f2cf01414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:34.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf4c00640 1 --2- 172.21.15.33:0/2063556442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf014ad00 0x7f2cf01414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf4c00640 1 -- 172.21.15.33:0/2063556442 >> v2:172.21.15.194:3300/0 conn(0x7f2cf00b53d0 msgr2=0x7f2cf014a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:34.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf4c00640 1 --2- 172.21.15.33:0/2063556442 >> v2:172.21.15.194:3300/0 conn(0x7f2cf00b53d0 0x7f2cf014a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf4c00640 1 -- 172.21.15.33:0/2063556442 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00b65f0 con 0x7f2cf00b3f90 2024-03-28T01:08:34.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf6000640 1 --2- 172.21.15.33:0/2063556442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf014ad00 0x7f2cf01414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:34.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.293+0000 7f2cf5600640 1 --2- 172.21.15.33:0/2063556442 >> v2:172.21.15.194:3300/0 conn(0x7f2cf00b53d0 0x7f2cf014a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:34.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.294+0000 7f2cf4c00640 1 --2- 172.21.15.33:0/2063556442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf00b3f90 0x7f2cf0140fb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00ba5e0 tx=0x7f2cdc02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0168ce470dcacc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:34.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.294+0000 7f2cefe00640 1 -- 172.21.15.33:0/2063556442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0317f0 con 0x7f2cf00b3f90 2024-03-28T01:08:34.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.294+0000 7f2cefe00640 1 -- 172.21.15.33:0/2063556442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc037070 con 0x7f2cf00b3f90 2024-03-28T01:08:34.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.294+0000 7f2cefe00640 1 -- 172.21.15.33:0/2063556442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc031bb0 con 0x7f2cf00b3f90 2024-03-28T01:08:34.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 -- 172.21.15.33:0/2063556442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf00b3f90 msgr2=0x7f2cf0140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 --2- 172.21.15.33:0/2063556442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf00b3f90 0x7f2cf0140fb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00ba5e0 tx=0x7f2cdc02b890 comp rx=0 tx=0).stop 2024-03-28T01:08:34.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 -- 172.21.15.33:0/2063556442 shutdown_connections 2024-03-28T01:08:34.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 --2- 172.21.15.33:0/2063556442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf014ad00 0x7f2cf01414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 --2- 172.21.15.33:0/2063556442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf00b3f90 0x7f2cf0140fb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 --2- 172.21.15.33:0/2063556442 >> v2:172.21.15.194:3300/0 conn(0x7f2cf00b53d0 0x7f2cf014a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 -- 172.21.15.33:0/2063556442 >> 172.21.15.33:0/2063556442 conn(0x7f2cf00bcf50 msgr2=0x7f2cf00bd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:34.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 -- 172.21.15.33:0/2063556442 shutdown_connections 2024-03-28T01:08:34.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.295+0000 7f2cf6a00640 1 -- 172.21.15.33:0/2063556442 wait complete. 2024-03-28T01:08:34.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.297+0000 7f2cf6a00640 1 Processor -- start 2024-03-28T01:08:34.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.297+0000 7f2cf6a00640 1 -- start start 2024-03-28T01:08:34.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf00b3f90 0x7f2cf0142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2cf00b53d0 0x7f2cf01dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf014ad00 0x7f2cf01e0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00b5930 con 0x7f2cf014ad00 2024-03-28T01:08:34.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00b57b0 con 0x7f2cf00b53d0 2024-03-28T01:08:34.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00b6d60 con 0x7f2cf00b3f90 2024-03-28T01:08:34.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf014ad00 0x7f2cf01e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf014ad00 0x7f2cf01e0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59344/0 (socket says 172.21.15.33:59344) 2024-03-28T01:08:34.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf6000640 1 -- 172.21.15.33:0/4157118482 learned_addr learned my addr 172.21.15.33:0/4157118482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:34.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf4c00640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2cf00b53d0 0x7f2cf01dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.298+0000 7f2cf5600640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf00b3f90 0x7f2cf0142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6000640 1 -- 172.21.15.33:0/4157118482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf00b3f90 msgr2=0x7f2cf0142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6000640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2cf00b3f90 0x7f2cf0142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6000640 1 -- 172.21.15.33:0/4157118482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2cf00b53d0 msgr2=0x7f2cf01dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6000640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2cf00b53d0 0x7f2cf01dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6000640 1 -- 172.21.15.33:0/4157118482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0144140 con 0x7f2cf014ad00 2024-03-28T01:08:34.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf4c00640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2cf00b53d0 0x7f2cf01dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:34.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6000640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf014ad00 0x7f2cf01e0300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800ccd0 tx=0x7f2ce800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:34.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800f680 con 0x7f2cf014ad00 2024-03-28T01:08:34.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce800f840 con 0x7f2cf014ad00 2024-03-28T01:08:34.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800fc30 con 0x7f2cf014ad00 2024-03-28T01:08:34.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf00aa1e0 con 0x7f2cf014ad00 2024-03-28T01:08:34.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.299+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf0143700 con 0x7f2cf014ad00 2024-03-28T01:08:34.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.300+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cf0075310 con 0x7f2cf014ad00 2024-03-28T01:08:34.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.302+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2ce80213f0 con 0x7f2cf014ad00 2024-03-28T01:08:34.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.303+0000 7f2ced600640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2cd006bbd0 0x7f2cd006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.303+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f2ce808bc90 con 0x7f2cf014ad00 2024-03-28T01:08:34.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.303+0000 7f2cf5600640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2cd006bbd0 0x7f2cd006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.303+0000 7f2ced600640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7f2cd0074080 0x7f2cd0076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.303+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ce8020e50 con 0x7f2cd0074080 2024-03-28T01:08:34.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.303+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce808c120 con 0x7f2cf014ad00 2024-03-28T01:08:34.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.303+0000 7f2cf4c00640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7f2cd0074080 0x7f2cd0076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.304+0000 7f2cf5600640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2cd006bbd0 0x7f2cd006e090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4006b10 tx=0x7f2ce4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:34.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.304+0000 7f2cf4c00640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7f2cd0074080 0x7f2cd0076520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:34.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.305+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== osd.3 v2:172.21.15.33:6800/2717396394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2ce8020e50 con 0x7f2cd0074080 2024-03-28T01:08:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 --> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2cf0148170 con 0x7f2cd0074080 2024-03-28T01:08:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2ced600640 1 -- 172.21.15.33:0/4157118482 <== osd.3 v2:172.21.15.33:6800/2717396394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2cf0148170 con 0x7f2cd0074080 2024-03-28T01:08:34.323 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:34.323 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:34.323 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7f2cd0074080 msgr2=0x7f2cd0076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6a00640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6800/2717396394,v1:172.21.15.33:6801/2717396394] conn(0x7f2cd0074080 0x7f2cd0076520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2cd006bbd0 msgr2=0x7f2cd006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6a00640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f2cd006bbd0 0x7f2cd006e090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4006b10 tx=0x7f2ce4008040 comp rx=0 tx=0).stop 2024-03-28T01:08:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf014ad00 msgr2=0x7f2cf01e0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6a00640 1 --2- 172.21.15.33:0/4157118482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf014ad00 0x7f2cf01e0300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800ccd0 tx=0x7f2ce800a650 comp rx=0 tx=0).stop 2024-03-28T01:08:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.320+0000 7f2cf6000640 1 -- 172.21.15.33:0/4157118482 reap_dead start 2024-03-28T01:08:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.321+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 shutdown_connections 2024-03-28T01:08:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.321+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 >> 172.21.15.33:0/4157118482 conn(0x7f2cf00bcf50 msgr2=0x7f2cf00b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.321+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 shutdown_connections 2024-03-28T01:08:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.321+0000 7f2cf6a00640 1 -- 172.21.15.33:0/4157118482 wait complete. 2024-03-28T01:08:34.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-28T01:08:34.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.547+0000 7f14a2400640 1 Processor -- start 2024-03-28T01:08:34.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.547+0000 7f14a2400640 1 -- start start 2024-03-28T01:08:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.548+0000 7f14a2400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 0x7f149c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.548+0000 7f14a2400640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149c0b0e70 con 0x7f149c0b5550 2024-03-28T01:08:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.548+0000 7f14a2400640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149c0b5930 con 0x7f149c0727d0 2024-03-28T01:08:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.548+0000 7f14a2400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0b1110 con 0x7f149c141750 2024-03-28T01:08:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.548+0000 7f14a1000640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f149c0b5550 0x7f149c071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58398/0 (socket says 172.21.15.33:58398) 2024-03-28T01:08:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.548+0000 7f14a1000640 1 -- 172.21.15.33:0/286227613 learned_addr learned my addr 172.21.15.33:0/286227613 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f14a1a00640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 0x7f149c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800316202 0 0) 0x7f149c0b0e70 con 0x7f149c0b5550 2024-03-28T01:08:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1480003520 con 0x7f149c0b5550 2024-03-28T01:08:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990551163 0 0) 0x7f1480003520 con 0x7f149c0b5550 2024-03-28T01:08:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 msgr2=0x7f149c14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 0x7f149c14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 >> v1:172.21.15.194:6789/0 conn(0x7f149c0727d0 legacy=0x7f149c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0b13f0 con 0x7f149c0b5550 2024-03-28T01:08:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f148c0019f0 con 0x7f149c0b5550 2024-03-28T01:08:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 >> v1:172.21.15.33:6789/0 conn(0x7f149c0b5550 legacy=0x7f149c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.549+0000 7f149b400640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14800039e0 0x7f1480003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f149b400640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1480004800 0x7f1480004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f149b400640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 0x7f1480005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0b1110 con 0x7f14800039e0 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148000bb80 con 0x7f1480004800 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148000bd80 con 0x7f149c141750 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1000640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14800039e0 0x7f1480003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f149be00640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1480004800 0x7f1480004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1a00640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 0x7f1480005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1000640 1 -- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 msgr2=0x7f1480005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1000640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 0x7f1480005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1000640 1 -- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1480004800 msgr2=0x7f1480004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1000640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1480004800 0x7f1480004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1000640 1 -- 172.21.15.33:0/286227613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480003520 con 0x7f14800039e0 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.550+0000 7f14a1a00640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c141750 0x7f1480005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.551+0000 7f14a1000640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14800039e0 0x7f1480003e00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f148c00a790 tx=0x7f1480005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9651a1033927392 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00a3b0 con 0x7f14800039e0 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c034650 con 0x7f14800039e0 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f149b400640 1 -- 172.21.15.33:0/286227613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c034ac0 con 0x7f14800039e0 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f14a2400640 1 -- 172.21.15.33:0/286227613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14800039e0 msgr2=0x7f1480003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f14a2400640 1 --2- 172.21.15.33:0/286227613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14800039e0 0x7f1480003e00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f148c00a790 tx=0x7f1480005830 comp rx=0 tx=0).stop 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f14a1a00640 1 -- 172.21.15.33:0/286227613 reap_dead start 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f14a2400640 1 -- 172.21.15.33:0/286227613 shutdown_connections 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.552+0000 7f14a2400640 1 -- 172.21.15.33:0/286227613 >> 172.21.15.33:0/286227613 conn(0x7f149c0b63d0 msgr2=0x7f149c0b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.553+0000 7f14a2400640 1 -- 172.21.15.33:0/286227613 shutdown_connections 2024-03-28T01:08:34.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.553+0000 7f14a2400640 1 -- 172.21.15.33:0/286227613 wait complete. 2024-03-28T01:08:34.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.554+0000 7f14a2400640 1 Processor -- start 2024-03-28T01:08:34.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.555+0000 7f14a2400640 1 -- start start 2024-03-28T01:08:34.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.555+0000 7f14a2400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14800039e0 0x7f149c0af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.555+0000 7f14a2400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c1dd2c0 0x7f149c1dff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.555+0000 7f14a2400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149c1dd740 0x7f149c1e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.555+0000 7f14a2400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c14ac60 con 0x7f149c1dd740 2024-03-28T01:08:34.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.555+0000 7f14a2400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c14aae0 con 0x7f14800039e0 2024-03-28T01:08:34.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.555+0000 7f14a2400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c14ade0 con 0x7f149c1dd2c0 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149c1dd740 0x7f149c1e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149c1dd740 0x7f149c1e04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59372/0 (socket says 172.21.15.33:59372) 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 -- 172.21.15.33:0/351125977 learned_addr learned my addr 172.21.15.33:0/351125977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1000640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14800039e0 0x7f149c0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f149be00640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c1dd2c0 0x7f149c1dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 -- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c1dd2c0 msgr2=0x7f149c1dff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c1dd2c0 0x7f149c1dff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 -- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14800039e0 msgr2=0x7f149c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14800039e0 0x7f149c0af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 -- 172.21.15.33:0/351125977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c1dcb00 con 0x7f149c1dd740 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.556+0000 7f14a1a00640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149c1dd740 0x7f149c1e04a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1480004780 tx=0x7f1494010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.557+0000 7f1498c00640 1 -- 172.21.15.33:0/351125977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400f040 con 0x7f149c1dd740 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.557+0000 7f1498c00640 1 -- 172.21.15.33:0/351125977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149400e840 con 0x7f149c1dd740 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.557+0000 7f1498c00640 1 -- 172.21.15.33:0/351125977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400ec30 con 0x7f149c1dd740 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.557+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c145500 con 0x7f149c1dd740 2024-03-28T01:08:34.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.557+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c145220 con 0x7f149c1dd740 2024-03-28T01:08:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.558+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c075310 con 0x7f149c1dd740 2024-03-28T01:08:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.559+0000 7f1498c00640 1 -- 172.21.15.33:0/351125977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f149401c400 con 0x7f149c1dd740 2024-03-28T01:08:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.560+0000 7f1498c00640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f147806bbd0 0x7f147806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.560+0000 7f1498c00640 1 -- 172.21.15.33:0/351125977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f149408ae20 con 0x7f149c1dd740 2024-03-28T01:08:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.560+0000 7f14a1000640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f147806bbd0 0x7f147806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.561+0000 7f14a1000640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f147806bbd0 0x7f147806e090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f148c034000 tx=0x7f148c02f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:34.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.565+0000 7f1498c00640 1 -- 172.21.15.33:0/351125977 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f1494018030 con 0x7f149c1dd740 2024-03-28T01:08:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.767+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f149c070e30 con 0x7f147806bbd0 2024-03-28T01:08:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.768+0000 7f1498c00640 1 -- 172.21.15.33:0/351125977 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f149c070e30 con 0x7f147806bbd0 2024-03-28T01:08:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 3 to deep-scrub 2024-03-28T01:08:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f147806bbd0 msgr2=0x7f147806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f147806bbd0 0x7f147806e090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f148c034000 tx=0x7f148c02f000 comp rx=0 tx=0).stop 2024-03-28T01:08:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149c1dd740 msgr2=0x7f149c1e04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149c1dd740 0x7f149c1e04a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1480004780 tx=0x7f1494010c20 comp rx=0 tx=0).stop 2024-03-28T01:08:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 shutdown_connections 2024-03-28T01:08:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f147806bbd0 0x7f147806e090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149c1dd740 0x7f149c1e04a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f149c1dd2c0 0x7f149c1dff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 --2- 172.21.15.33:0/351125977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14800039e0 0x7f149c0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 >> 172.21.15.33:0/351125977 conn(0x7f149c0b63d0 msgr2=0x7f149c0705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 shutdown_connections 2024-03-28T01:08:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:34.771+0000 7f14a2400640 1 -- 172.21.15.33:0/351125977 wait complete. 2024-03-28T01:08:34.794 INFO:tasks.ceph:Scrubbing osd.4 2024-03-28T01:08:34.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:35.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.008+0000 7fc4cac00640 1 Processor -- start 2024-03-28T01:08:35.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.008+0000 7fc4cac00640 1 -- start start 2024-03-28T01:08:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4cac00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4c4072760 0x7fc4c40b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4cac00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 0x7fc4c413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4cac00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c4072b40 con 0x7fc4c413f340 2024-03-28T01:08:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4cac00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c40b6120 con 0x7fc4c4072760 2024-03-28T01:08:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4cac00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40b5ad0 con 0x7fc4c40b5370 2024-03-28T01:08:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4c9800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4c4072760 0x7fc4c40b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4c9800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4c4072760 0x7fc4c40b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59386/0 (socket says 172.21.15.33:59386) 2024-03-28T01:08:35.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4c9800640 1 -- 172.21.15.33:0/1213866125 learned_addr learned my addr 172.21.15.33:0/1213866125 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:35.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.010+0000 7fc4c8e00640 1 --2- 172.21.15.33:0/1213866125 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 0x7fc4c413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.011+0000 7fc4c9800640 1 -- 172.21.15.33:0/1213866125 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 msgr2=0x7fc4c413ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:35.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.011+0000 7fc4c9800640 1 --2- 172.21.15.33:0/1213866125 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 0x7fc4c413ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.011+0000 7fc4c9800640 1 -- 172.21.15.33:0/1213866125 >> v1:172.21.15.194:6789/0 conn(0x7fc4c413f340 legacy=0x7fc4c41451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.011+0000 7fc4c9800640 1 -- 172.21.15.33:0/1213866125 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c40b5db0 con 0x7fc4c4072760 2024-03-28T01:08:35.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.012+0000 7fc4c9800640 1 --2- 172.21.15.33:0/1213866125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c40b3400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18de69c2806974e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.012+0000 7fc4c3e00640 1 -- 172.21.15.33:0/1213866125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8035020 con 0x7fc4c4072760 2024-03-28T01:08:35.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.012+0000 7fc4c3e00640 1 -- 172.21.15.33:0/1213866125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8031640 con 0x7fc4c4072760 2024-03-28T01:08:35.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.012+0000 7fc4c3e00640 1 -- 172.21.15.33:0/1213866125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b802d040 con 0x7fc4c4072760 2024-03-28T01:08:35.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 -- 172.21.15.33:0/1213866125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 msgr2=0x7fc4c40b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 --2- 172.21.15.33:0/1213866125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c40b3400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b802b8f0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 -- 172.21.15.33:0/1213866125 shutdown_connections 2024-03-28T01:08:35.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 --2- 172.21.15.33:0/1213866125 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 0x7fc4c413ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 --2- 172.21.15.33:0/1213866125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c40b3400 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 -- 172.21.15.33:0/1213866125 >> 172.21.15.33:0/1213866125 conn(0x7fc4c40b6460 msgr2=0x7fc4c40b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:35.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 -- 172.21.15.33:0/1213866125 shutdown_connections 2024-03-28T01:08:35.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.013+0000 7fc4cac00640 1 -- 172.21.15.33:0/1213866125 wait complete. 2024-03-28T01:08:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.015+0000 7fc4cac00640 1 Processor -- start 2024-03-28T01:08:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.015+0000 7fc4cac00640 1 -- start start 2024-03-28T01:08:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4cac00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c413fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4cac00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 0x7fc4c41dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4cac00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c413f340 0x7fc4c4137d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4cac00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4145a70 con 0x7fc4c4072760 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4cac00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41458f0 con 0x7fc4c413f340 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4cac00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c406cba0 con 0x7fc4c40b5370 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4c9800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c413fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4c9800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c413fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59396/0 (socket says 172.21.15.33:59396) 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.016+0000 7fc4c9800640 1 -- 172.21.15.33:0/2016337693 learned_addr learned my addr 172.21.15.33:0/2016337693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c8e00640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 0x7fc4c41dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4ca200640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c413f340 0x7fc4c4137d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c9800640 1 -- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 msgr2=0x7fc4c41dfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c9800640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c40b5370 0x7fc4c41dfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c9800640 1 -- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c413f340 msgr2=0x7fc4c4137d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c9800640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c413f340 0x7fc4c4137d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c9800640 1 -- 172.21.15.33:0/2016337693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c406cdf0 con 0x7fc4c4072760 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c9800640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c413fba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802b930 tx=0x7fc4b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8040070 con 0x7fc4c4072760 2024-03-28T01:08:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b803e070 con 0x7fc4c4072760 2024-03-28T01:08:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8035040 con 0x7fc4c4072760 2024-03-28T01:08:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.017+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c4141100 con 0x7fc4c4072760 2024-03-28T01:08:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.018+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c406de00 con 0x7fc4c4072760 2024-03-28T01:08:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.019+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4c4075310 con 0x7fc4c4072760 2024-03-28T01:08:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.020+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc4b8038020 con 0x7fc4c4072760 2024-03-28T01:08:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.021+0000 7fc4c1600640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc4a406bc50 0x7fc4a406e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.021+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fc4b80ac460 con 0x7fc4c4072760 2024-03-28T01:08:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.022+0000 7fc4c8e00640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc4a406bc50 0x7fc4a406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.022+0000 7fc4c1600640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7fc4a4074190 0x7fc4a4076630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.022+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 --> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4b8001d30 con 0x7fc4a4074190 2024-03-28T01:08:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.022+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b80b3050 con 0x7fc4c4072760 2024-03-28T01:08:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.022+0000 7fc4c8e00640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc4a406bc50 0x7fc4a406e110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4c406e220 tx=0x7fc4b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.022+0000 7fc4ca200640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7fc4a4074190 0x7fc4a4076630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.023+0000 7fc4ca200640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7fc4a4074190 0x7fc4a4076630 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.025+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== osd.4 v2:172.21.15.194:6802/548012972 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc4b8001d30 con 0x7fc4a4074190 2024-03-28T01:08:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.040+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 --> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc4c40732d0 con 0x7fc4a4074190 2024-03-28T01:08:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.041+0000 7fc4c1600640 1 -- 172.21.15.33:0/2016337693 <== osd.4 v2:172.21.15.194:6802/548012972 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc4c40732d0 con 0x7fc4a4074190 2024-03-28T01:08:35.044 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:35.044 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:35.044 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:35.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.041+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7fc4a4074190 msgr2=0x7fc4a4076630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.041+0000 7fc4cac00640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.194:6802/548012972,v1:172.21.15.194:6804/548012972] conn(0x7fc4a4074190 0x7fc4a4076630 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.041+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc4a406bc50 msgr2=0x7fc4a406e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.041+0000 7fc4cac00640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fc4a406bc50 0x7fc4a406e110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4c406e220 tx=0x7fc4b0008040 comp rx=0 tx=0).stop 2024-03-28T01:08:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.042+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 msgr2=0x7fc4c413fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.042+0000 7fc4cac00640 1 --2- 172.21.15.33:0/2016337693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4c4072760 0x7fc4c413fba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802b930 tx=0x7fc4b80046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.042+0000 7fc4ca200640 1 -- 172.21.15.33:0/2016337693 reap_dead start 2024-03-28T01:08:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.042+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 shutdown_connections 2024-03-28T01:08:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.042+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 >> 172.21.15.33:0/2016337693 conn(0x7fc4c40b6460 msgr2=0x7fc4c40b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.042+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 shutdown_connections 2024-03-28T01:08:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.042+0000 7fc4cac00640 1 -- 172.21.15.33:0/2016337693 wait complete. 2024-03-28T01:08:35.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-28T01:08:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.279+0000 7fbe6b000640 1 Processor -- start 2024-03-28T01:08:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.280+0000 7fbe6b000640 1 -- start start 2024-03-28T01:08:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe6b000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 0x7fbe64141390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe6b000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe640bd470 con 0x7fbe640bd090 2024-03-28T01:08:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe6b000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe640b3f90 con 0x7fbe6414fa80 2024-03-28T01:08:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe6b000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b6630 con 0x7fbe64140f30 2024-03-28T01:08:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe69c00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbe640bd090 0x7fbe6414a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58402/0 (socket says 172.21.15.33:58402) 2024-03-28T01:08:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe69c00640 1 -- 172.21.15.33:0/3830360165 learned_addr learned my addr 172.21.15.33:0/3830360165 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe69200640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 0x7fbe64141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.281+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997433753 0 0) 0x7fbe640bd470 con 0x7fbe640bd090 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe4c003520 con 0x7fbe640bd090 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3583074304 0 0) 0x7fbe4c003520 con 0x7fbe640bd090 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 msgr2=0x7fbe64141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 0x7fbe64141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 >> v1:172.21.15.194:6789/0 conn(0x7fbe6414fa80 legacy=0x7fbe64151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640b4450 con 0x7fbe640bd090 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7fbe58001670 con 0x7fbe640bd090 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 >> v1:172.21.15.33:6789/0 conn(0x7fbe640bd090 legacy=0x7fbe6414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe4c0039e0 0x7fbe4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 0x7fbe4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe4c004d40 0x7fbe4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b6630 con 0x7fbe4c004d40 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c00ba70 con 0x7fbe4c0039e0 2024-03-28T01:08:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.282+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c00bc70 con 0x7fbe64140f30 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe6a600640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe4c004d40 0x7fbe4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe69c00640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe4c0039e0 0x7fbe4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe69200640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 0x7fbe4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe6a600640 1 -- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 msgr2=0x7fbe4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe6a600640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe64140f30 0x7fbe4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe6a600640 1 -- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe4c0039e0 msgr2=0x7fbe4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe6a600640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe4c0039e0 0x7fbe4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.283+0000 7fbe6a600640 1 -- 172.21.15.33:0/3830360165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c003520 con 0x7fbe4c004d40 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.284+0000 7fbe6a600640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe4c004d40 0x7fbe4c0051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b9560 tx=0x7fbe4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c44967504d32488 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.284+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c001a70 con 0x7fbe4c004d40 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.284+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c003b70 con 0x7fbe4c004d40 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.284+0000 7fbe63e00640 1 -- 172.21.15.33:0/3830360165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0050d0 con 0x7fbe4c004d40 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.285+0000 7fbe6b000640 1 -- 172.21.15.33:0/3830360165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe4c004d40 msgr2=0x7fbe4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.285+0000 7fbe6b000640 1 --2- 172.21.15.33:0/3830360165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe4c004d40 0x7fbe4c0051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b9560 tx=0x7fbe4c005830 comp rx=0 tx=0).stop 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.285+0000 7fbe6a600640 1 -- 172.21.15.33:0/3830360165 reap_dead start 2024-03-28T01:08:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.285+0000 7fbe6b000640 1 -- 172.21.15.33:0/3830360165 shutdown_connections 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.285+0000 7fbe6b000640 1 -- 172.21.15.33:0/3830360165 >> 172.21.15.33:0/3830360165 conn(0x7fbe64072940 msgr2=0x7fbe64072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.285+0000 7fbe6b000640 1 -- 172.21.15.33:0/3830360165 shutdown_connections 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.285+0000 7fbe6b000640 1 -- 172.21.15.33:0/3830360165 wait complete. 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.287+0000 7fbe6b000640 1 Processor -- start 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.287+0000 7fbe6b000640 1 -- start start 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe6b000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe4c004d40 0x7fbe6413ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe6b000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe641497a0 0x7fbe64142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe6b000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe64142c40 0x7fbe641dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe6b000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b4730 con 0x7fbe641497a0 2024-03-28T01:08:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe6b000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b5400 con 0x7fbe64142c40 2024-03-28T01:08:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe6b000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b6290 con 0x7fbe4c004d40 2024-03-28T01:08:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe641497a0 0x7fbe64142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe641497a0 0x7fbe64142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59420/0 (socket says 172.21.15.33:59420) 2024-03-28T01:08:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 -- 172.21.15.33:0/3383198030 learned_addr learned my addr 172.21.15.33:0/3383198030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69c00640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe4c004d40 0x7fbe6413ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 -- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe4c004d40 msgr2=0x7fbe6413ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe4c004d40 0x7fbe6413ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 -- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe64142c40 msgr2=0x7fbe641dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe6a600640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe64142c40 0x7fbe641dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe64142c40 0x7fbe641dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.288+0000 7fbe69200640 1 -- 172.21.15.33:0/3383198030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe64149eb0 con 0x7fbe641497a0 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.289+0000 7fbe69200640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe641497a0 0x7fbe64142700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbe50011600 tx=0x7fbe50011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.289+0000 7fbe61600640 1 -- 172.21.15.33:0/3383198030 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000f920 con 0x7fbe641497a0 2024-03-28T01:08:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.289+0000 7fbe61600640 1 -- 172.21.15.33:0/3383198030 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5000fac0 con 0x7fbe641497a0 2024-03-28T01:08:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.289+0000 7fbe61600640 1 -- 172.21.15.33:0/3383198030 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000fe70 con 0x7fbe641497a0 2024-03-28T01:08:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.289+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe640b9ea0 con 0x7fbe641497a0 2024-03-28T01:08:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.289+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe64148bd0 con 0x7fbe641497a0 2024-03-28T01:08:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.291+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe64148010 con 0x7fbe641497a0 2024-03-28T01:08:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.292+0000 7fbe61600640 1 -- 172.21.15.33:0/3383198030 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbe50015460 con 0x7fbe641497a0 2024-03-28T01:08:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.292+0000 7fbe61600640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbe4406bbd0 0x7fbe4406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.293+0000 7fbe61600640 1 -- 172.21.15.33:0/3383198030 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fbe5008b240 con 0x7fbe641497a0 2024-03-28T01:08:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.293+0000 7fbe69c00640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbe4406bbd0 0x7fbe4406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.293+0000 7fbe69c00640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbe4406bbd0 0x7fbe4406e090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe58001eb0 tx=0x7fbe58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.306+0000 7fbe61600640 1 -- 172.21.15.33:0/3383198030 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbe64148010 con 0x7fbe641497a0 2024-03-28T01:08:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.570+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe64075310 con 0x7fbe4406bbd0 2024-03-28T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.570+0000 7fbe61600640 1 -- 172.21.15.33:0/3383198030 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbe64075310 con 0x7fbe4406bbd0 2024-03-28T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 4 to deep-scrub 2024-03-28T01:08:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbe4406bbd0 msgr2=0x7fbe4406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbe4406bbd0 0x7fbe4406e090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe58001eb0 tx=0x7fbe58009040 comp rx=0 tx=0).stop 2024-03-28T01:08:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe641497a0 msgr2=0x7fbe64142700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe641497a0 0x7fbe64142700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbe50011600 tx=0x7fbe50011ac0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 shutdown_connections 2024-03-28T01:08:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fbe4406bbd0 0x7fbe4406e090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe64142c40 0x7fbe641dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe641497a0 0x7fbe64142700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 --2- 172.21.15.33:0/3383198030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe4c004d40 0x7fbe6413ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 >> 172.21.15.33:0/3383198030 conn(0x7fbe64072940 msgr2=0x7fbe640b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.574+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 shutdown_connections 2024-03-28T01:08:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.575+0000 7fbe6b000640 1 -- 172.21.15.33:0/3383198030 wait complete. 2024-03-28T01:08:35.602 INFO:tasks.ceph:Scrubbing osd.5 2024-03-28T01:08:35.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.837+0000 7f9db7200640 1 Processor -- start 2024-03-28T01:08:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.837+0000 7f9db7200640 1 -- start start 2024-03-28T01:08:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.838+0000 7f9db7200640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f9db806f1d0 0x7f9db813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.838+0000 7f9db7200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db80b2620 con 0x7f9db813f2d0 2024-03-28T01:08:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.838+0000 7f9db7200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db806cf70 con 0x7f9db80b2240 2024-03-28T01:08:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.838+0000 7f9db7200640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db806be00 con 0x7f9db806f1d0 2024-03-28T01:08:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.839+0000 7f9db6800640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9db813f2d0 0x7f9db814f7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58404/0 (socket says 172.21.15.33:58404) 2024-03-28T01:08:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.839+0000 7f9db6800640 1 -- 172.21.15.33:0/4121221644 learned_addr learned my addr 172.21.15.33:0/4121221644 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:35.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.839+0000 7f9db5400640 1 --2- 172.21.15.33:0/4121221644 >> v2:172.21.15.194:3300/0 conn(0x7f9db806f1d0 0x7f9db813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.839+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234220986 0 0) 0x7f9db80b2620 con 0x7f9db813f2d0 2024-03-28T01:08:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.839+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d9c003520 con 0x7f9db813f2d0 2024-03-28T01:08:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.839+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3633467969 0 0) 0x7f9d9c003520 con 0x7f9db813f2d0 2024-03-28T01:08:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 >> v2:172.21.15.194:3300/0 conn(0x7f9db806f1d0 msgr2=0x7f9db813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 --2- 172.21.15.33:0/4121221644 >> v2:172.21.15.194:3300/0 conn(0x7f9db806f1d0 0x7f9db813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 >> v1:172.21.15.194:6789/0 conn(0x7f9db80b2240 legacy=0x7f9db8144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db806b600 con 0x7f9db813f2d0 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f9db0001670 con 0x7f9db813f2d0 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 >> v1:172.21.15.33:6789/0 conn(0x7f9db813f2d0 legacy=0x7f9db814f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c0039b0 0x7f9d9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9d9c004800 0x7f9d9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d9c0051c0 0x7f9d9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db806be00 con 0x7f9d9c0039b0 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c00c160 con 0x7f9d9c0051c0 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c00c360 con 0x7f9d9c004800 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db5e00640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c0039b0 0x7f9d9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db6800640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d9c0051c0 0x7f9d9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db5e00640 1 -- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9d9c004800 msgr2=0x7f9d9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db5e00640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9d9c004800 0x7f9d9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db5e00640 1 -- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d9c0051c0 msgr2=0x7f9d9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db5e00640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d9c0051c0 0x7f9d9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db5e00640 1 -- 172.21.15.33:0/4121221644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c003520 con 0x7f9d9c0039b0 2024-03-28T01:08:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db6800640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d9c0051c0 0x7f9d9c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.840+0000 7f9db6800640 1 -- 172.21.15.33:0/4121221644 reap_dead start 2024-03-28T01:08:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.842+0000 7f9db5e00640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c0039b0 0x7f9d9c003dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001750 tx=0x7f9d9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beba6cf887f107 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.842+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0001370 con 0x7f9d9c0039b0 2024-03-28T01:08:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.842+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da00052f0 con 0x7f9d9c0039b0 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.842+0000 7f9db4a00640 1 -- 172.21.15.33:0/4121221644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0003850 con 0x7f9d9c0039b0 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.843+0000 7f9db7200640 1 -- 172.21.15.33:0/4121221644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c0039b0 msgr2=0x7f9d9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.843+0000 7f9db7200640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c0039b0 0x7f9d9c003dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001750 tx=0x7f9d9c005830 comp rx=0 tx=0).stop 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.843+0000 7f9db7200640 1 -- 172.21.15.33:0/4121221644 shutdown_connections 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.843+0000 7f9db7200640 1 --2- 172.21.15.33:0/4121221644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c0039b0 0x7f9d9c003dd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.843+0000 7f9db7200640 1 -- 172.21.15.33:0/4121221644 >> 172.21.15.33:0/4121221644 conn(0x7f9db80b55a0 msgr2=0x7f9db80b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.843+0000 7f9db7200640 1 -- 172.21.15.33:0/4121221644 shutdown_connections 2024-03-28T01:08:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.843+0000 7f9db7200640 1 -- 172.21.15.33:0/4121221644 wait complete. 2024-03-28T01:08:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.846+0000 7f9db7200640 1 Processor -- start 2024-03-28T01:08:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.846+0000 7f9db7200640 1 -- start start 2024-03-28T01:08:35.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db7200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9d9c0039b0 0x7f9db81423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9db8141800 0x7f9db81dcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db81dd130 0x7f9db813bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db7200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db806cd30 con 0x7f9db81dd130 2024-03-28T01:08:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db806cbb0 con 0x7f9db8141800 2024-03-28T01:08:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db7200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db806b030 con 0x7f9d9c0039b0 2024-03-28T01:08:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db6800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db81dd130 0x7f9db813bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db6800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db81dd130 0x7f9db813bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59432/0 (socket says 172.21.15.33:59432) 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.847+0000 7f9db6800640 1 -- 172.21.15.33:0/2377096796 learned_addr learned my addr 172.21.15.33:0/2377096796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db5e00640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9d9c0039b0 0x7f9db81423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db5400640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9db8141800 0x7f9db81dcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db6800640 1 -- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9d9c0039b0 msgr2=0x7f9db81423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db6800640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9d9c0039b0 0x7f9db81423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db6800640 1 -- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9db8141800 msgr2=0x7f9db81dcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db6800640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9db8141800 0x7f9db81dcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db6800640 1 -- 172.21.15.33:0/2377096796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db8140390 con 0x7f9db81dd130 2024-03-28T01:08:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9db6800640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db81dd130 0x7f9db813bb10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9db813f6b0 tx=0x7f9db000d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.848+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00074e0 con 0x7f9db81dd130 2024-03-28T01:08:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.849+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00047e0 con 0x7f9db81dd130 2024-03-28T01:08:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.849+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0004b90 con 0x7f9db81dd130 2024-03-28T01:08:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.849+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db81dd5b0 con 0x7f9db81dd130 2024-03-28T01:08:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.849+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db81408f0 con 0x7f9db81dd130 2024-03-28T01:08:35.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.850+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9db8075310 con 0x7f9db81dd130 2024-03-28T01:08:35.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.851+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9db00133d0 con 0x7f9db81dd130 2024-03-28T01:08:35.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.852+0000 7f9dae000640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9d9406bbd0 0x7f9d9406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.853+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9db008a7d0 con 0x7f9db81dd130 2024-03-28T01:08:35.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.853+0000 7f9db5e00640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9d9406bbd0 0x7f9d9406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.853+0000 7f9dae000640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f9d94074080 0x7f9d94076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:35.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.853+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 --> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d94076bf0 con 0x7f9d94074080 2024-03-28T01:08:35.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.853+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db008f2c0 con 0x7f9db81dd130 2024-03-28T01:08:35.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.854+0000 7f9db5400640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f9d94074080 0x7f9d94076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:35.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.854+0000 7f9db5e00640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9d9406bbd0 0x7f9d9406e090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9da0033000 tx=0x7f9da0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.855+0000 7f9db5400640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f9d94074080 0x7f9d94076520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.857+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== osd.5 v2:172.21.15.194:6816/3192443429 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9d94076bf0 con 0x7f9d94074080 2024-03-28T01:08:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.871+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 --> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9db80b3ff0 con 0x7f9d94074080 2024-03-28T01:08:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.872+0000 7f9dae000640 1 -- 172.21.15.33:0/2377096796 <== osd.5 v2:172.21.15.194:6816/3192443429 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9db80b3ff0 con 0x7f9d94074080 2024-03-28T01:08:35.875 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:35.875 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:35.875 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f9d94074080 msgr2=0x7f9d94076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.194:6816/3192443429,v1:172.21.15.194:6817/3192443429] conn(0x7f9d94074080 0x7f9d94076520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9d9406bbd0 msgr2=0x7f9d9406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9d9406bbd0 0x7f9d9406e090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9da0033000 tx=0x7f9da0009030 comp rx=0 tx=0).stop 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db81dd130 msgr2=0x7f9db813bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 --2- 172.21.15.33:0/2377096796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db81dd130 0x7f9db813bb10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9db813f6b0 tx=0x7f9db000d950 comp rx=0 tx=0).stop 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db6800640 1 -- 172.21.15.33:0/2377096796 reap_dead start 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 shutdown_connections 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 >> 172.21.15.33:0/2377096796 conn(0x7f9db80b55a0 msgr2=0x7f9db813df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 shutdown_connections 2024-03-28T01:08:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:35.873+0000 7f9db7200640 1 -- 172.21.15.33:0/2377096796 wait complete. 2024-03-28T01:08:35.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-28T01:08:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.117+0000 7fa618000640 1 Processor -- start 2024-03-28T01:08:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.117+0000 7fa618000640 1 -- start start 2024-03-28T01:08:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.119+0000 7fa618000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa61006d3e0 0x7fa6100b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.119+0000 7fa618000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7fa6100b4fa0 0x7fa61013f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.119+0000 7fa618000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61006d7c0 con 0x7fa6100b2d00 2024-03-28T01:08:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.119+0000 7fa618000640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100b10f0 con 0x7fa61006d3e0 2024-03-28T01:08:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.119+0000 7fa618000640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100b0aa0 con 0x7fa6100b4fa0 2024-03-28T01:08:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.119+0000 7fa615600640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa61006d3e0 0x7fa6100b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.120+0000 7fa615600640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa61006d3e0 0x7fa6100b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59442/0 (socket says 172.21.15.33:59442) 2024-03-28T01:08:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.120+0000 7fa615600640 1 -- 172.21.15.33:0/790367690 learned_addr learned my addr 172.21.15.33:0/790367690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.120+0000 7fa616000640 1 --2- 172.21.15.33:0/790367690 >> v2:172.21.15.194:3300/0 conn(0x7fa6100b4fa0 0x7fa61013f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.120+0000 7fa615600640 1 -- 172.21.15.33:0/790367690 >> v2:172.21.15.194:3300/0 conn(0x7fa6100b4fa0 msgr2=0x7fa61013f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.120+0000 7fa615600640 1 --2- 172.21.15.33:0/790367690 >> v2:172.21.15.194:3300/0 conn(0x7fa6100b4fa0 0x7fa61013f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.120+0000 7fa615600640 1 -- 172.21.15.33:0/790367690 >> v1:172.21.15.194:6789/0 conn(0x7fa6100b2d00 legacy=0x7fa61013eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.120+0000 7fa615600640 1 -- 172.21.15.33:0/790367690 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100b0d80 con 0x7fa61006d3e0 2024-03-28T01:08:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.121+0000 7fa615600640 1 --2- 172.21.15.33:0/790367690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa6100b4a60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa6100b1bf0 tx=0x7fa60402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8c4d095ffbeb4c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.121+0000 7fa60fe00640 1 -- 172.21.15.33:0/790367690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60402e070 con 0x7fa61006d3e0 2024-03-28T01:08:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.121+0000 7fa60fe00640 1 -- 172.21.15.33:0/790367690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604037070 con 0x7fa61006d3e0 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.121+0000 7fa60fe00640 1 -- 172.21.15.33:0/790367690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604031a00 con 0x7fa61006d3e0 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 -- 172.21.15.33:0/790367690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 msgr2=0x7fa6100b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 --2- 172.21.15.33:0/790367690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa6100b4a60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa6100b1bf0 tx=0x7fa60402b880 comp rx=0 tx=0).stop 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 -- 172.21.15.33:0/790367690 shutdown_connections 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 --2- 172.21.15.33:0/790367690 >> v2:172.21.15.194:3300/0 conn(0x7fa6100b4fa0 0x7fa61013f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 --2- 172.21.15.33:0/790367690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa6100b4a60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 -- 172.21.15.33:0/790367690 >> 172.21.15.33:0/790367690 conn(0x7fa61006b030 msgr2=0x7fa61006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 -- 172.21.15.33:0/790367690 shutdown_connections 2024-03-28T01:08:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.122+0000 7fa618000640 1 -- 172.21.15.33:0/790367690 wait complete. 2024-03-28T01:08:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.124+0000 7fa618000640 1 Processor -- start 2024-03-28T01:08:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.124+0000 7fa618000640 1 -- start start 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa618000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa610071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa618000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6100b2d00 0x7fa6101de120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa618000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6100b4fa0 0x7fa6101de660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa618000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100b0150 con 0x7fa61006d3e0 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa618000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100affd0 con 0x7fa6100b4fa0 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa618000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61014fab0 con 0x7fa6100b2d00 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa615600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa610071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa615600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa610071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59444/0 (socket says 172.21.15.33:59444) 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa615600640 1 -- 172.21.15.33:0/878657673 learned_addr learned my addr 172.21.15.33:0/878657673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa616000640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6100b4fa0 0x7fa6101de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.125+0000 7fa614c00640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6100b2d00 0x7fa6101de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa615600640 1 -- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6100b2d00 msgr2=0x7fa6101de120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa615600640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6100b2d00 0x7fa6101de120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa615600640 1 -- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6100b4fa0 msgr2=0x7fa6101de660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa615600640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6100b4fa0 0x7fa6101de660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa615600640 1 -- 172.21.15.33:0/878657673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61009c8c0 con 0x7fa61006d3e0 2024-03-28T01:08:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa615600640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa610071b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa604000c00 tx=0x7fa6040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa60d600640 1 -- 172.21.15.33:0/878657673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604032880 con 0x7fa61006d3e0 2024-03-28T01:08:36.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa60d600640 1 -- 172.21.15.33:0/878657673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604032a20 con 0x7fa61006d3e0 2024-03-28T01:08:36.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa60d600640 1 -- 172.21.15.33:0/878657673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604032e10 con 0x7fa61006d3e0 2024-03-28T01:08:36.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.126+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61009d960 con 0x7fa61006d3e0 2024-03-28T01:08:36.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.127+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa610072690 con 0x7fa61006d3e0 2024-03-28T01:08:36.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.128+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa610075380 con 0x7fa61006d3e0 2024-03-28T01:08:36.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.129+0000 7fa60d600640 1 -- 172.21.15.33:0/878657673 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa60404c060 con 0x7fa61006d3e0 2024-03-28T01:08:36.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.130+0000 7fa60d600640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa5f006bbd0 0x7fa5f006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.131+0000 7fa60d600640 1 -- 172.21.15.33:0/878657673 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fa6040ac390 con 0x7fa61006d3e0 2024-03-28T01:08:36.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.131+0000 7fa614c00640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa5f006bbd0 0x7fa5f006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.131+0000 7fa614c00640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa5f006bbd0 0x7fa5f006e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6101dbce0 tx=0x7fa5fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.135+0000 7fa60d600640 1 -- 172.21.15.33:0/878657673 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa604040070 con 0x7fa61006d3e0 2024-03-28T01:08:36.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.344+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6101435a0 con 0x7fa5f006bbd0 2024-03-28T01:08:36.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.345+0000 7fa60d600640 1 -- 172.21.15.33:0/878657673 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa6101435a0 con 0x7fa5f006bbd0 2024-03-28T01:08:36.347 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 5 to deep-scrub 2024-03-28T01:08:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa5f006bbd0 msgr2=0x7fa5f006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa5f006bbd0 0x7fa5f006e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6101dbce0 tx=0x7fa5fc009040 comp rx=0 tx=0).stop 2024-03-28T01:08:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 msgr2=0x7fa610071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa610071b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa604000c00 tx=0x7fa6040046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 shutdown_connections 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fa5f006bbd0 0x7fa5f006e090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6100b4fa0 0x7fa6101de660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6100b2d00 0x7fa6101de120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 --2- 172.21.15.33:0/878657673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa61006d3e0 0x7fa610071b00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.347+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 >> 172.21.15.33:0/878657673 conn(0x7fa61006b030 msgr2=0x7fa6100b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.348+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 shutdown_connections 2024-03-28T01:08:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.348+0000 7fa618000640 1 -- 172.21.15.33:0/878657673 wait complete. 2024-03-28T01:08:36.373 INFO:tasks.ceph:Scrubbing osd.6 2024-03-28T01:08:36.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.587+0000 7febfe400640 1 Processor -- start 2024-03-28T01:08:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.588+0000 7febfe400640 1 -- start start 2024-03-28T01:08:36.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.589+0000 7febfe400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf80b2240 0x7febf8144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.589+0000 7febfe400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf813f4a0 0x7febf814f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.589+0000 7febfe400640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf80b2620 con 0x7febf806f1d0 2024-03-28T01:08:36.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.589+0000 7febfe400640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf806be00 con 0x7febf80b2240 2024-03-28T01:08:36.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.589+0000 7febfe400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf806cf70 con 0x7febf813f4a0 2024-03-28T01:08:36.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfd000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf80b2240 0x7febf8144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfd000640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf80b2240 0x7febf8144ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59448/0 (socket says 172.21.15.33:59448) 2024-03-28T01:08:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfd000640 1 -- 172.21.15.33:0/483544517 learned_addr learned my addr 172.21.15.33:0/483544517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfda00640 1 --2- 172.21.15.33:0/483544517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf813f4a0 0x7febf814f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfd000640 1 -- 172.21.15.33:0/483544517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf813f4a0 msgr2=0x7febf814f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfd000640 1 --2- 172.21.15.33:0/483544517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf813f4a0 0x7febf814f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfd000640 1 -- 172.21.15.33:0/483544517 >> v1:172.21.15.194:6789/0 conn(0x7febf806f1d0 legacy=0x7febf813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.590+0000 7febfd000640 1 -- 172.21.15.33:0/483544517 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf806b600 con 0x7febf80b2240 2024-03-28T01:08:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.591+0000 7febfd000640 1 --2- 172.21.15.33:0/483544517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf80b2240 0x7febf8144ec0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febe0009870 tx=0x7febe002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e37de9e5338d09 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febf7400640 1 -- 172.21.15.33:0/483544517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00317c0 con 0x7febf80b2240 2024-03-28T01:08:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febf7400640 1 -- 172.21.15.33:0/483544517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe0037070 con 0x7febf80b2240 2024-03-28T01:08:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febf7400640 1 -- 172.21.15.33:0/483544517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0031b80 con 0x7febf80b2240 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 -- 172.21.15.33:0/483544517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf80b2240 msgr2=0x7febf8144ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 --2- 172.21.15.33:0/483544517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf80b2240 0x7febf8144ec0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febe0009870 tx=0x7febe002b860 comp rx=0 tx=0).stop 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 -- 172.21.15.33:0/483544517 shutdown_connections 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 --2- 172.21.15.33:0/483544517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf813f4a0 0x7febf814f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 --2- 172.21.15.33:0/483544517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf80b2240 0x7febf8144ec0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 -- 172.21.15.33:0/483544517 >> 172.21.15.33:0/483544517 conn(0x7febf80b55a0 msgr2=0x7febf80b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 -- 172.21.15.33:0/483544517 shutdown_connections 2024-03-28T01:08:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.592+0000 7febfe400640 1 -- 172.21.15.33:0/483544517 wait complete. 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.594+0000 7febfe400640 1 Processor -- start 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.594+0000 7febfe400640 1 -- start start 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfe400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf806f1d0 0x7febf80b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfe400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf80b2240 0x7febf81e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfe400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf813f4a0 0x7febf81e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfe400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf806cd30 con 0x7febf813f4a0 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfe400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf806cbb0 con 0x7febf80b2240 2024-03-28T01:08:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfe400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf806b030 con 0x7febf806f1d0 2024-03-28T01:08:36.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfda00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf813f4a0 0x7febf81e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfda00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf813f4a0 0x7febf81e0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59462/0 (socket says 172.21.15.33:59462) 2024-03-28T01:08:36.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.595+0000 7febfda00640 1 -- 172.21.15.33:0/588817405 learned_addr learned my addr 172.21.15.33:0/588817405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:36.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfd000640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf806f1d0 0x7febf80b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febf7e00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf80b2240 0x7febf81e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfda00640 1 -- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf806f1d0 msgr2=0x7febf80b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfda00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febf806f1d0 0x7febf80b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfda00640 1 -- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf80b2240 msgr2=0x7febf81e04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfda00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf80b2240 0x7febf81e04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfda00640 1 -- 172.21.15.33:0/588817405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf8146720 con 0x7febf813f4a0 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febf7e00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf80b2240 0x7febf81e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfda00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf813f4a0 0x7febf81e0a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febf0007e10 tx=0x7febf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0014620 con 0x7febf813f4a0 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf0019070 con 0x7febf813f4a0 2024-03-28T01:08:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0014a20 con 0x7febf813f4a0 2024-03-28T01:08:36.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.596+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf8141bf0 con 0x7febf813f4a0 2024-03-28T01:08:36.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.597+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf8146020 con 0x7febf813f4a0 2024-03-28T01:08:36.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.597+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7febf8075310 con 0x7febf813f4a0 2024-03-28T01:08:36.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.599+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7febf0014bc0 con 0x7febf813f4a0 2024-03-28T01:08:36.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.599+0000 7febf4c00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7febd406bbd0 0x7febd406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.600+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7febf008b3f0 con 0x7febf813f4a0 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.600+0000 7febfd000640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7febd406bbd0 0x7febd406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.600+0000 7febf4c00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7febd4074080 0x7febd4076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.600+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 --> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febf000ee90 con 0x7febd4074080 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.600+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febf0092050 con 0x7febf813f4a0 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.600+0000 7febfd000640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7febd406bbd0 0x7febd406e090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7febe0036000 tx=0x7febe003d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.601+0000 7febf7e00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7febd4074080 0x7febd4076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.601+0000 7febf7e00640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7febd4074080 0x7febd4076520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.603+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== osd.6 v2:172.21.15.194:6800/1743959830 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7febf000ee90 con 0x7febd4074080 2024-03-28T01:08:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.618+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 --> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7febf81e0110 con 0x7febd4074080 2024-03-28T01:08:36.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.619+0000 7febf4c00640 1 -- 172.21.15.33:0/588817405 <== osd.6 v2:172.21.15.194:6800/1743959830 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7febf81e0110 con 0x7febd4074080 2024-03-28T01:08:36.622 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:36.622 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:36.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.619+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7febd4074080 msgr2=0x7febd4076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.619+0000 7febfe400640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.194:6800/1743959830,v1:172.21.15.194:6801/1743959830] conn(0x7febd4074080 0x7febd4076520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7febd406bbd0 msgr2=0x7febd406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7febd406bbd0 0x7febd406e090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7febe0036000 tx=0x7febe003d040 comp rx=0 tx=0).stop 2024-03-28T01:08:36.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf813f4a0 msgr2=0x7febf81e0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 --2- 172.21.15.33:0/588817405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf813f4a0 0x7febf81e0a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febf0007e10 tx=0x7febf000e4b0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfda00640 1 -- 172.21.15.33:0/588817405 reap_dead start 2024-03-28T01:08:36.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 shutdown_connections 2024-03-28T01:08:36.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 >> 172.21.15.33:0/588817405 conn(0x7febf80b55a0 msgr2=0x7febf813df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:36.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 shutdown_connections 2024-03-28T01:08:36.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.620+0000 7febfe400640 1 -- 172.21.15.33:0/588817405 wait complete. 2024-03-28T01:08:36.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-28T01:08:36.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.847+0000 7f3f5d800640 1 Processor -- start 2024-03-28T01:08:36.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.847+0000 7f3f5d800640 1 -- start start 2024-03-28T01:08:36.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.848+0000 7f3f5d800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f3f580b55e0 0x7f3f58148530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.848+0000 7f3f5d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f580707a0 0x7f3f58148a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.848+0000 7f3f5d800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f580b59c0 con 0x7f3f5813ec40 2024-03-28T01:08:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.848+0000 7f3f5d800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f58072760 con 0x7f3f580b55e0 2024-03-28T01:08:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.848+0000 7f3f5d800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580b12e0 con 0x7f3f580707a0 2024-03-28T01:08:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.849+0000 7f3f5ce00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3f5813ec40 0x7f3f5813f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58412/0 (socket says 172.21.15.33:58412) 2024-03-28T01:08:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.849+0000 7f3f5ce00640 1 -- 172.21.15.33:0/4235334126 learned_addr learned my addr 172.21.15.33:0/4235334126 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.849+0000 7f3f4f400640 1 --2- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f580707a0 0x7f3f58148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.849+0000 7f3f57e00640 1 --2- 172.21.15.33:0/4235334126 >> v2:172.21.15.194:3300/0 conn(0x7f3f580b55e0 0x7f3f58148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.849+0000 7f3f57400640 1 -- 172.21.15.33:0/4235334126 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476478884 0 0) 0x7f3f580b59c0 con 0x7f3f5813ec40 2024-03-28T01:08:36.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.849+0000 7f3f57400640 1 -- 172.21.15.33:0/4235334126 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f38003520 con 0x7f3f5813ec40 2024-03-28T01:08:36.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.850+0000 7f3f57e00640 1 -- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f580707a0 msgr2=0x7f3f58148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.850+0000 7f3f57e00640 1 --2- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f580707a0 0x7f3f58148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.850+0000 7f3f57e00640 1 -- 172.21.15.33:0/4235334126 >> v1:172.21.15.33:6789/0 conn(0x7f3f5813ec40 legacy=0x7f3f5813f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.850+0000 7f3f57e00640 1 -- 172.21.15.33:0/4235334126 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58071f60 con 0x7f3f580b55e0 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.850+0000 7f3f4f400640 1 --2- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f580707a0 0x7f3f58148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.851+0000 7f3f57e00640 1 --2- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580b55e0 0x7f3f58148530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f480097a0 tx=0x7f3f4802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644c9e4b3327d1ce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.851+0000 7f3f57400640 1 -- 172.21.15.33:0/4235334126 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48038070 con 0x7f3f580b55e0 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.852+0000 7f3f57400640 1 -- 172.21.15.33:0/4235334126 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48031700 con 0x7f3f580b55e0 2024-03-28T01:08:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.852+0000 7f3f57400640 1 -- 172.21.15.33:0/4235334126 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48031ad0 con 0x7f3f580b55e0 2024-03-28T01:08:36.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.852+0000 7f3f5d800640 1 -- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580b55e0 msgr2=0x7f3f58148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.852+0000 7f3f5d800640 1 --2- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580b55e0 0x7f3f58148530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f480097a0 tx=0x7f3f4802b7a0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.853+0000 7f3f5d800640 1 -- 172.21.15.33:0/4235334126 shutdown_connections 2024-03-28T01:08:36.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.853+0000 7f3f5d800640 1 --2- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f580707a0 0x7f3f58148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.853+0000 7f3f5d800640 1 --2- 172.21.15.33:0/4235334126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580b55e0 0x7f3f58148530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.853+0000 7f3f5d800640 1 -- 172.21.15.33:0/4235334126 >> 172.21.15.33:0/4235334126 conn(0x7f3f580b6460 msgr2=0x7f3f580b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.853+0000 7f3f5d800640 1 -- 172.21.15.33:0/4235334126 shutdown_connections 2024-03-28T01:08:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.853+0000 7f3f5d800640 1 -- 172.21.15.33:0/4235334126 wait complete. 2024-03-28T01:08:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.855+0000 7f3f5d800640 1 Processor -- start 2024-03-28T01:08:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.855+0000 7f3f5d800640 1 -- start start 2024-03-28T01:08:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f5d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580707a0 0x7f3f581435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f5d800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f580b55e0 0x7f3f581d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f5d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f5813ec40 0x7f3f580b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f5d800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58070e00 con 0x7f3f580b55e0 2024-03-28T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f5d800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58070c80 con 0x7f3f580707a0 2024-03-28T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f5d800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5806c140 con 0x7f3f5813ec40 2024-03-28T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f4f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f580b55e0 0x7f3f581d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f4f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f580b55e0 0x7f3f581d88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59468/0 (socket says 172.21.15.33:59468) 2024-03-28T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.856+0000 7f3f4f400640 1 -- 172.21.15.33:0/1220180691 learned_addr learned my addr 172.21.15.33:0/1220180691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f57e00640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580707a0 0x7f3f581435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f5ce00640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f5813ec40 0x7f3f580b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f4f400640 1 -- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f5813ec40 msgr2=0x7f3f580b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f4f400640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f5813ec40 0x7f3f580b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f4f400640 1 -- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580707a0 msgr2=0x7f3f581435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f4f400640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580707a0 0x7f3f581435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f4f400640 1 -- 172.21.15.33:0/1220180691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58145cf0 con 0x7f3f580b55e0 2024-03-28T01:08:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f5ce00640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f5813ec40 0x7f3f580b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f57e00640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580707a0 0x7f3f581435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f4f400640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f580b55e0 0x7f3f581d88a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ebe0 tx=0x7f3f4000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.857+0000 7f3f54c00640 1 -- 172.21.15.33:0/1220180691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40017070 con 0x7f3f580b55e0 2024-03-28T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.858+0000 7f3f54c00640 1 -- 172.21.15.33:0/1220180691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4000cbd0 con 0x7f3f580b55e0 2024-03-28T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.858+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5813fad0 con 0x7f3f580b55e0 2024-03-28T01:08:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.858+0000 7f3f54c00640 1 -- 172.21.15.33:0/1220180691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40013470 con 0x7f3f580b55e0 2024-03-28T01:08:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.858+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58145300 con 0x7f3f580b55e0 2024-03-28T01:08:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.859+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f58075310 con 0x7f3f580b55e0 2024-03-28T01:08:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.860+0000 7f3f54c00640 1 -- 172.21.15.33:0/1220180691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3f4001a070 con 0x7f3f580b55e0 2024-03-28T01:08:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.861+0000 7f3f54c00640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3f3406bbd0 0x7f3f3406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.861+0000 7f3f54c00640 1 -- 172.21.15.33:0/1220180691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f3f4008a9f0 con 0x7f3f580b55e0 2024-03-28T01:08:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.861+0000 7f3f57e00640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3f3406bbd0 0x7f3f3406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.862+0000 7f3f57e00640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3f3406bbd0 0x7f3f3406e090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f48035000 tx=0x7f3f48008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:36.870+0000 7f3f54c00640 1 -- 172.21.15.33:0/1220180691 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3f40059490 con 0x7f3f580b55e0 2024-03-28T01:08:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.073+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f5806e1f0 con 0x7f3f3406bbd0 2024-03-28T01:08:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.074+0000 7f3f54c00640 1 -- 172.21.15.33:0/1220180691 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3f5806e1f0 con 0x7f3f3406bbd0 2024-03-28T01:08:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 6 to deep-scrub 2024-03-28T01:08:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.076+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3f3406bbd0 msgr2=0x7f3f3406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.076+0000 7f3f5d800640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3f3406bbd0 0x7f3f3406e090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f48035000 tx=0x7f3f48008030 comp rx=0 tx=0).stop 2024-03-28T01:08:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.076+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f580b55e0 msgr2=0x7f3f581d88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.076+0000 7f3f5d800640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f580b55e0 0x7f3f581d88a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ebe0 tx=0x7f3f4000c520 comp rx=0 tx=0).stop 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 shutdown_connections 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f3f3406bbd0 0x7f3f3406e090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f5813ec40 0x7f3f580b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f580b55e0 0x7f3f581d88a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 --2- 172.21.15.33:0/1220180691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f580707a0 0x7f3f581435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 >> 172.21.15.33:0/1220180691 conn(0x7f3f580b6460 msgr2=0x7f3f58148920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 shutdown_connections 2024-03-28T01:08:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.077+0000 7f3f5d800640 1 -- 172.21.15.33:0/1220180691 wait complete. 2024-03-28T01:08:37.105 INFO:tasks.ceph:Scrubbing osd.7 2024-03-28T01:08:37.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-28T01:08:37.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.322+0000 7f5781c00640 1 Processor -- start 2024-03-28T01:08:37.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.322+0000 7f5781c00640 1 -- start start 2024-03-28T01:08:37.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5781c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c06c7d0 0x7f577c140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5781c00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f577c141470 0x7f577c14a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5781c00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c14b0e0 con 0x7f577c14ac90 2024-03-28T01:08:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5781c00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c1418b0 con 0x7f577c141470 2024-03-28T01:08:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5781c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c06cbb0 con 0x7f577c06c7d0 2024-03-28T01:08:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5773400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f577c141470 0x7f577c14a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5773400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f577c141470 0x7f577c14a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59470/0 (socket says 172.21.15.33:59470) 2024-03-28T01:08:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.325+0000 7f5773400640 1 -- 172.21.15.33:0/231862495 learned_addr learned my addr 172.21.15.33:0/231862495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.326+0000 7f577be00640 1 --2- 172.21.15.33:0/231862495 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c06c7d0 0x7f577c140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.326+0000 7f5773400640 1 -- 172.21.15.33:0/231862495 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c06c7d0 msgr2=0x7f577c140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.326+0000 7f5773400640 1 --2- 172.21.15.33:0/231862495 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c06c7d0 0x7f577c140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.326+0000 7f5773400640 1 -- 172.21.15.33:0/231862495 >> v1:172.21.15.194:6789/0 conn(0x7f577c14ac90 legacy=0x7f577c06b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.326+0000 7f5773400640 1 -- 172.21.15.33:0/231862495 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0b22b0 con 0x7f577c141470 2024-03-28T01:08:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.327+0000 7f5773400640 1 --2- 172.21.15.33:0/231862495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c141470 0x7f577c14a750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5764009860 tx=0x7f576402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ee01a397a5ea8a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.328+0000 7f577b400640 1 -- 172.21.15.33:0/231862495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764034070 con 0x7f577c141470 2024-03-28T01:08:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.328+0000 7f577b400640 1 -- 172.21.15.33:0/231862495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5764030840 con 0x7f577c141470 2024-03-28T01:08:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.328+0000 7f577b400640 1 -- 172.21.15.33:0/231862495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764030c10 con 0x7f577c141470 2024-03-28T01:08:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 -- 172.21.15.33:0/231862495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c141470 msgr2=0x7f577c14a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 --2- 172.21.15.33:0/231862495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c141470 0x7f577c14a750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5764009860 tx=0x7f576402b980 comp rx=0 tx=0).stop 2024-03-28T01:08:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 -- 172.21.15.33:0/231862495 shutdown_connections 2024-03-28T01:08:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 --2- 172.21.15.33:0/231862495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c141470 0x7f577c14a750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 --2- 172.21.15.33:0/231862495 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c06c7d0 0x7f577c140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 -- 172.21.15.33:0/231862495 >> 172.21.15.33:0/231862495 conn(0x7f577c0b9c10 msgr2=0x7f577c0ba020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 -- 172.21.15.33:0/231862495 shutdown_connections 2024-03-28T01:08:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.329+0000 7f5781c00640 1 -- 172.21.15.33:0/231862495 wait complete. 2024-03-28T01:08:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.331+0000 7f5781c00640 1 Processor -- start 2024-03-28T01:08:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.331+0000 7f5781c00640 1 -- start start 2024-03-28T01:08:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f5781c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c06c7d0 0x7f577c147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f5781c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c141470 0x7f577c070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f5781c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f577c14ac90 0x7f577c1dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f5781c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0b51a0 con 0x7f577c06c7d0 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f5781c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0b4ec0 con 0x7f577c14ac90 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f5781c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0b5480 con 0x7f577c141470 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f577be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c06c7d0 0x7f577c147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f577be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c06c7d0 0x7f577c147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59484/0 (socket says 172.21.15.33:59484) 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.332+0000 7f577be00640 1 -- 172.21.15.33:0/991926082 learned_addr learned my addr 172.21.15.33:0/991926082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f5781200640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f577c14ac90 0x7f577c1dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f5773400640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c141470 0x7f577c070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f577be00640 1 -- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c141470 msgr2=0x7f577c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f577be00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f577c141470 0x7f577c070430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f577be00640 1 -- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f577c14ac90 msgr2=0x7f577c1dcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f577be00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f577c14ac90 0x7f577c1dcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f577be00640 1 -- 172.21.15.33:0/991926082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0719d0 con 0x7f577c06c7d0 2024-03-28T01:08:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f5781200640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f577c14ac90 0x7f577c1dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f577be00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c06c7d0 0x7f577c147880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d800 tx=0x7f576c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c00b6c0 con 0x7f577c06c7d0 2024-03-28T01:08:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c019070 con 0x7f577c06c7d0 2024-03-28T01:08:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.333+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c0af430 con 0x7f577c06c7d0 2024-03-28T01:08:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.334+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c00ba80 con 0x7f577c06c7d0 2024-03-28T01:08:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.334+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c0b0690 con 0x7f577c06c7d0 2024-03-28T01:08:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.335+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f577c0716f0 con 0x7f577c06c7d0 2024-03-28T01:08:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.336+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f576c00bc20 con 0x7f577c06c7d0 2024-03-28T01:08:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.337+0000 7f5778c00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f575806bbd0 0x7f575806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.337+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f576c08afb0 con 0x7f577c06c7d0 2024-03-28T01:08:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.337+0000 7f5773400640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f575806bbd0 0x7f575806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.338+0000 7f5778c00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7f5758074090 0x7f5758076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.338+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f576c017e50 con 0x7f5758074090 2024-03-28T01:08:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.338+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f576c092050 con 0x7f577c06c7d0 2024-03-28T01:08:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.338+0000 7f5781200640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7f5758074090 0x7f5758076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.338+0000 7f5773400640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f575806bbd0 0x7f575806e090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5764036000 tx=0x7f576403b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:37.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.339+0000 7f5781200640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7f5758074090 0x7f5758076530 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:37.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.341+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== osd.7 v2:172.21.15.194:6824/2849955959 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f576c017e50 con 0x7f5758074090 2024-03-28T01:08:37.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.356+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 --> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f577c075310 con 0x7f5758074090 2024-03-28T01:08:37.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5778c00640 1 -- 172.21.15.33:0/991926082 <== osd.7 v2:172.21.15.194:6824/2849955959 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f577c075310 con 0x7f5758074090 2024-03-28T01:08:37.359 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-03-28T01:08:37.360 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T01:08:37.360 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7f5758074090 msgr2=0x7f5758076530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.194:6824/2849955959,v1:172.21.15.194:6825/2849955959] conn(0x7f5758074090 0x7f5758076530 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f575806bbd0 msgr2=0x7f575806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f575806bbd0 0x7f575806e090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5764036000 tx=0x7f576403b040 comp rx=0 tx=0).stop 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c06c7d0 msgr2=0x7f577c147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 --2- 172.21.15.33:0/991926082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f577c06c7d0 0x7f577c147880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d800 tx=0x7f576c00dcc0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781200640 1 -- 172.21.15.33:0/991926082 reap_dead start 2024-03-28T01:08:37.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 shutdown_connections 2024-03-28T01:08:37.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 >> 172.21.15.33:0/991926082 conn(0x7f577c0b9c10 msgr2=0x7f577c13f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:37.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.357+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 shutdown_connections 2024-03-28T01:08:37.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.358+0000 7f5781c00640 1 -- 172.21.15.33:0/991926082 wait complete. 2024-03-28T01:08:37.378 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-28T01:08:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.587+0000 7fde16200640 1 Processor -- start 2024-03-28T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.587+0000 7fde16200640 1 -- start start 2024-03-28T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.588+0000 7fde16200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde100bd3a0 0x7fde10141140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.588+0000 7fde16200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100bd780 con 0x7fde10141680 2024-03-28T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.588+0000 7fde16200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100b68b0 con 0x7fde1014b020 2024-03-28T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.588+0000 7fde16200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde100b5870 con 0x7fde100bd3a0 2024-03-28T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.588+0000 7fde0f400640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde100bd3a0 0x7fde10141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.588+0000 7fde0ea00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fde10141680 0x7fde1014a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58422/0 (socket says 172.21.15.33:58422) 2024-03-28T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.589+0000 7fde0ea00640 1 -- 172.21.15.33:0/665426967 learned_addr learned my addr 172.21.15.33:0/665426967 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.589+0000 7fde0e000640 1 -- 172.21.15.33:0/665426967 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327205081 0 0) 0x7fde100bd780 con 0x7fde10141680 2024-03-28T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.589+0000 7fde0f400640 1 -- 172.21.15.33:0/665426967 >> v1:172.21.15.194:6789/0 conn(0x7fde1014b020 legacy=0x7fde10151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.589+0000 7fde0f400640 1 -- 172.21.15.33:0/665426967 >> v1:172.21.15.33:6789/0 conn(0x7fde10141680 legacy=0x7fde1014a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.589+0000 7fde0f400640 1 -- 172.21.15.33:0/665426967 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100b4430 con 0x7fde100bd3a0 2024-03-28T01:08:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.591+0000 7fde0f400640 1 --2- 172.21.15.33:0/665426967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde100bd3a0 0x7fde10141140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fde000097a0 tx=0x7fde0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e62dab5de9b741 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.591+0000 7fde0e000640 1 -- 172.21.15.33:0/665426967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000315a0 con 0x7fde100bd3a0 2024-03-28T01:08:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.591+0000 7fde0e000640 1 -- 172.21.15.33:0/665426967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00037070 con 0x7fde100bd3a0 2024-03-28T01:08:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.591+0000 7fde0e000640 1 -- 172.21.15.33:0/665426967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00031960 con 0x7fde100bd3a0 2024-03-28T01:08:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.592+0000 7fde16200640 1 -- 172.21.15.33:0/665426967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde100bd3a0 msgr2=0x7fde10141140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.592+0000 7fde16200640 1 --2- 172.21.15.33:0/665426967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde100bd3a0 0x7fde10141140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fde000097a0 tx=0x7fde0002b840 comp rx=0 tx=0).stop 2024-03-28T01:08:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.592+0000 7fde16200640 1 -- 172.21.15.33:0/665426967 shutdown_connections 2024-03-28T01:08:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.592+0000 7fde16200640 1 --2- 172.21.15.33:0/665426967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde100bd3a0 0x7fde10141140 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.592+0000 7fde16200640 1 -- 172.21.15.33:0/665426967 >> 172.21.15.33:0/665426967 conn(0x7fde10072b80 msgr2=0x7fde10072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.592+0000 7fde16200640 1 -- 172.21.15.33:0/665426967 shutdown_connections 2024-03-28T01:08:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.592+0000 7fde16200640 1 -- 172.21.15.33:0/665426967 wait complete. 2024-03-28T01:08:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.594+0000 7fde16200640 1 Processor -- start 2024-03-28T01:08:37.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.594+0000 7fde16200640 1 -- start start 2024-03-28T01:08:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde16200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde100bd3a0 0x7fde101e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde16200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde10141680 0x7fde10148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde16200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde1014b020 0x7fde101489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde16200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100b4ba0 con 0x7fde10141680 2024-03-28T01:08:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde16200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100b64f0 con 0x7fde1014b020 2024-03-28T01:08:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde16200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde101e09a0 con 0x7fde100bd3a0 2024-03-28T01:08:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde0ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde10141680 0x7fde10148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde0ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde10141680 0x7fde10148470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59508/0 (socket says 172.21.15.33:59508) 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde0ea00640 1 -- 172.21.15.33:0/2665554465 learned_addr learned my addr 172.21.15.33:0/2665554465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde0ea00640 1 -- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde100bd3a0 msgr2=0x7fde101e00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde0fe00640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde1014b020 0x7fde101489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.596+0000 7fde0ea00640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde100bd3a0 0x7fde101e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde0ea00640 1 -- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde1014b020 msgr2=0x7fde101489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde0ea00640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde1014b020 0x7fde101489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde0ea00640 1 -- 172.21.15.33:0/2665554465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde101448e0 con 0x7fde10141680 2024-03-28T01:08:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde0fe00640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde1014b020 0x7fde101489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde0ea00640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde10141680 0x7fde10148470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c4d0 tx=0x7fddfc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde07400640 1 -- 172.21.15.33:0/2665554465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc018070 con 0x7fde10141680 2024-03-28T01:08:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde07400640 1 -- 172.21.15.33:0/2665554465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc007500 con 0x7fde10141680 2024-03-28T01:08:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde07400640 1 -- 172.21.15.33:0/2665554465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc004a20 con 0x7fde10141680 2024-03-28T01:08:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde10143940 con 0x7fde10141680 2024-03-28T01:08:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.597+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde101435d0 con 0x7fde10141680 2024-03-28T01:08:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.598+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde10075380 con 0x7fde10141680 2024-03-28T01:08:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.599+0000 7fde07400640 1 -- 172.21.15.33:0/2665554465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fddfc01b070 con 0x7fde10141680 2024-03-28T01:08:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.601+0000 7fde07400640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddf006bc50 0x7fddf006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.601+0000 7fde07400640 1 -- 172.21.15.33:0/2665554465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fddfc089f60 con 0x7fde10141680 2024-03-28T01:08:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.601+0000 7fde0f400640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddf006bc50 0x7fddf006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.602+0000 7fde0f400640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddf006bc50 0x7fddf006e110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde00036000 tx=0x7fde00030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.605+0000 7fde07400640 1 -- 172.21.15.33:0/2665554465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fddfc0589e0 con 0x7fde10141680 2024-03-28T01:08:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.814+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fde10144cd0 con 0x7fddf006bc50 2024-03-28T01:08:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.815+0000 7fde07400640 1 -- 172.21.15.33:0/2665554465 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fde10144cd0 con 0x7fddf006bc50 2024-03-28T01:08:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 7 to deep-scrub 2024-03-28T01:08:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.817+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddf006bc50 msgr2=0x7fddf006e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.817+0000 7fde16200640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddf006bc50 0x7fddf006e110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde00036000 tx=0x7fde00030000 comp rx=0 tx=0).stop 2024-03-28T01:08:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.817+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde10141680 msgr2=0x7fde10148470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.817+0000 7fde16200640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde10141680 0x7fde10148470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c4d0 tx=0x7fddfc00c990 comp rx=0 tx=0).stop 2024-03-28T01:08:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 shutdown_connections 2024-03-28T01:08:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fddf006bc50 0x7fddf006e110 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde1014b020 0x7fde101489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde10141680 0x7fde10148470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 --2- 172.21.15.33:0/2665554465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde100bd3a0 0x7fde101e00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 >> 172.21.15.33:0/2665554465 conn(0x7fde10072b80 msgr2=0x7fde10147320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 shutdown_connections 2024-03-28T01:08:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:37.818+0000 7fde16200640 1 -- 172.21.15.33:0/2665554465 wait complete. 2024-03-28T01:08:37.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:08:38.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.102+0000 7f352cc00640 1 Processor -- start 2024-03-28T01:08:38.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.102+0000 7f352cc00640 1 -- start start 2024-03-28T01:08:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f352cc00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f35280b2130 0x7f352813edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f352cc00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3528148cf0 0x7f352814f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f352cc00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35280b2510 con 0x7f352813f330 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f352cc00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35280b4670 con 0x7f3528148cf0 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f352cc00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352806bef0 con 0x7f35280b2130 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f351ea00640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f352813f330 0x7f35281485e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58434/0 (socket says 172.21.15.33:58434) 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f351ea00640 1 -- 172.21.15.33:0/1991560429 learned_addr learned my addr 172.21.15.33:0/1991560429 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.104+0000 7f3527e00640 1 --2- 172.21.15.33:0/1991560429 >> v2:172.21.15.33:3300/0 conn(0x7f3528148cf0 0x7f352814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.105+0000 7f3527400640 1 --2- 172.21.15.33:0/1991560429 >> v2:172.21.15.194:3300/0 conn(0x7f35280b2130 0x7f352813edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.105+0000 7f3527e00640 1 -- 172.21.15.33:0/1991560429 >> v2:172.21.15.194:3300/0 conn(0x7f35280b2130 msgr2=0x7f352813edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.105+0000 7f3527e00640 1 --2- 172.21.15.33:0/1991560429 >> v2:172.21.15.194:3300/0 conn(0x7f35280b2130 0x7f352813edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.105+0000 7f3527e00640 1 -- 172.21.15.33:0/1991560429 >> v1:172.21.15.33:6789/0 conn(0x7f352813f330 legacy=0x7f35281485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.105+0000 7f3527e00640 1 -- 172.21.15.33:0/1991560429 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280b4c40 con 0x7f3528148cf0 2024-03-28T01:08:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.105+0000 7f3526a00640 1 -- 172.21.15.33:0/1991560429 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581982705 0 0) 0x7f35280b2510 con 0x7f352813f330 2024-03-28T01:08:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.106+0000 7f3527e00640 1 --2- 172.21.15.33:0/1991560429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3528148cf0 0x7f352814f850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f35280b29a0 tx=0x7f352002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b75a841a51ed416 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.106+0000 7f3526a00640 1 -- 172.21.15.33:0/1991560429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520038070 con 0x7f3528148cf0 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.106+0000 7f3526a00640 1 -- 172.21.15.33:0/1991560429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3520031710 con 0x7f3528148cf0 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.106+0000 7f3526a00640 1 -- 172.21.15.33:0/1991560429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520031ae0 con 0x7f3528148cf0 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 -- 172.21.15.33:0/1991560429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3528148cf0 msgr2=0x7f352814f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 --2- 172.21.15.33:0/1991560429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3528148cf0 0x7f352814f850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f35280b29a0 tx=0x7f352002b7b0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 -- 172.21.15.33:0/1991560429 shutdown_connections 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 --2- 172.21.15.33:0/1991560429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3528148cf0 0x7f352814f850 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 --2- 172.21.15.33:0/1991560429 >> v2:172.21.15.194:3300/0 conn(0x7f35280b2130 0x7f352813edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 -- 172.21.15.33:0/1991560429 >> 172.21.15.33:0/1991560429 conn(0x7f35280b89c0 msgr2=0x7f35280b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:38.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 -- 172.21.15.33:0/1991560429 shutdown_connections 2024-03-28T01:08:38.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.107+0000 7f352cc00640 1 -- 172.21.15.33:0/1991560429 wait complete. 2024-03-28T01:08:38.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.109+0000 7f352cc00640 1 Processor -- start 2024-03-28T01:08:38.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.109+0000 7f352cc00640 1 -- start start 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f352cc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35280b2130 0x7f352813d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f352cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352813f330 0x7f3528143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f352cc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3528148cf0 0x7f35281dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f352cc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528072d20 con 0x7f35280b2130 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f352cc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35280b4ec0 con 0x7f352813f330 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f352cc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3528143460 con 0x7f3528148cf0 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35280b2130 0x7f352813d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35280b2130 0x7f352813d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59534/0 (socket says 172.21.15.33:59534) 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 -- 172.21.15.33:0/148971753 learned_addr learned my addr 172.21.15.33:0/148971753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 -- 172.21.15.33:0/148971753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3528148cf0 msgr2=0x7f35281dcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3528148cf0 0x7f35281dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 -- 172.21.15.33:0/148971753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352813f330 msgr2=0x7f3528143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352813f330 0x7f3528143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.110+0000 7f3527400640 1 -- 172.21.15.33:0/148971753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528142e50 con 0x7f35280b2130 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.111+0000 7f3527400640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35280b2130 0x7f352813d4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3518007e40 tx=0x7f351800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.111+0000 7f351fe00640 1 -- 172.21.15.33:0/148971753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518014620 con 0x7f35280b2130 2024-03-28T01:08:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.111+0000 7f351fe00640 1 -- 172.21.15.33:0/148971753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3518019070 con 0x7f35280b2130 2024-03-28T01:08:38.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.111+0000 7f351fe00640 1 -- 172.21.15.33:0/148971753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518014a00 con 0x7f35280b2130 2024-03-28T01:08:38.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.111+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35281441b0 con 0x7f35280b2130 2024-03-28T01:08:38.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.111+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3528142ac0 con 0x7f35280b2130 2024-03-28T01:08:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.112+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3528075310 con 0x7f35280b2130 2024-03-28T01:08:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.113+0000 7f351fe00640 1 -- 172.21.15.33:0/148971753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3518014ba0 con 0x7f35280b2130 2024-03-28T01:08:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.114+0000 7f351fe00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f350406bbd0 0x7f350406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.114+0000 7f351fe00640 1 -- 172.21.15.33:0/148971753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f351808b400 con 0x7f35280b2130 2024-03-28T01:08:38.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.114+0000 7f351ea00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f350406bbd0 0x7f350406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:38.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.115+0000 7f351ea00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f350406bbd0 0x7f350406e090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3510007fd0 tx=0x7f3510007920 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.119+0000 7f351fe00640 1 -- 172.21.15.33:0/148971753 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3518059e00 con 0x7f35280b2130 2024-03-28T01:08:38.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.320+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f352806b470 con 0x7f350406bbd0 2024-03-28T01:08:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.339+0000 7f351fe00640 1 -- 172.21.15.33:0/148971753 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250834 (secure 0 0 0) 0x7f352806b470 con 0x7f350406bbd0 2024-03-28T01:08:38.345 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:08:38.347 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:08:38.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f350406bbd0 msgr2=0x7f350406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:38.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f350406bbd0 0x7f350406e090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3510007fd0 tx=0x7f3510007920 comp rx=0 tx=0).stop 2024-03-28T01:08:38.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35280b2130 msgr2=0x7f352813d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:38.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35280b2130 0x7f352813d4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3518007e40 tx=0x7f351800e4b0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 shutdown_connections 2024-03-28T01:08:38.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f350406bbd0 0x7f350406e090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3528148cf0 0x7f35281dcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352813f330 0x7f3528143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 --2- 172.21.15.33:0/148971753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35280b2130 0x7f352813d4f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:38.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 >> 172.21.15.33:0/148971753 conn(0x7f35280b89c0 msgr2=0x7f35280b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:38.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 shutdown_connections 2024-03-28T01:08:38.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:38.346+0000 7f352cc00640 1 -- 172.21.15.33:0/148971753 wait complete. 2024-03-28T01:08:38.366 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-03-28T01:08:36.430439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":741183,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":125,"num_read_kb":109,"num_write":358,"num_write_kb":2667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1335,"ondisk_log_size":1335,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219592,"kb_used_data":3464,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755128,"statfs":{"total":773094113280,"available":772869251072,"internally_reserved":0,"allocated":3547136,"data_stored":2198102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":81934,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":71,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003932"},"pg_stats":[{"pgid":"3.3f","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.148877+0000","last_change":"2024-03-28T01:06:22.748846+0000","last_active":"2024-03-28T01:08:27.148877+0000","last_peered":"2024-03-28T01:08:27.148877+0000","last_clean":"2024-03-28T01:08:27.148877+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:08:27.148877+0000","last_undegraded":"2024-03-28T01:08:27.148877+0000","last_fullsized":"2024-03-28T01:08:27.148877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:03:27.339812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.460173+0000","last_change":"2024-03-28T01:06:22.732451+0000","last_active":"2024-03-28T01:08:30.460173+0000","last_peered":"2024-03-28T01:08:30.460173+0000","last_clean":"2024-03-28T01:08:30.460173+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:08:30.460173+0000","last_undegraded":"2024-03-28T01:08:30.460173+0000","last_fullsized":"2024-03-28T01:08:30.460173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'15","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.457185+0000","last_change":"2024-03-28T01:06:22.718788+0000","last_active":"2024-03-28T01:08:30.457185+0000","last_peered":"2024-03-28T01:08:30.457185+0000","last_clean":"2024-03-28T01:08:30.457185+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:08:30.457185+0000","last_undegraded":"2024-03-28T01:08:30.457185+0000","last_fullsized":"2024-03-28T01:08:30.457185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'16","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163483+0000","last_change":"2024-03-28T01:06:22.751092+0000","last_active":"2024-03-28T01:08:27.163483+0000","last_peered":"2024-03-28T01:08:27.163483+0000","last_clean":"2024-03-28T01:08:27.163483+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:08:27.163483+0000","last_undegraded":"2024-03-28T01:08:27.163483+0000","last_fullsized":"2024-03-28T01:08:27.163483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.151104+0000","last_change":"2024-03-28T01:06:22.746114+0000","last_active":"2024-03-28T01:08:27.151104+0000","last_peered":"2024-03-28T01:08:27.151104+0000","last_clean":"2024-03-28T01:08:27.151104+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:08:27.151104+0000","last_undegraded":"2024-03-28T01:08:27.151104+0000","last_fullsized":"2024-03-28T01:08:27.151104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:31:35.811621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.158052+0000","last_change":"2024-03-28T01:06:22.736600+0000","last_active":"2024-03-28T01:08:27.158052+0000","last_peered":"2024-03-28T01:08:27.158052+0000","last_clean":"2024-03-28T01:08:27.158052+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:08:27.158052+0000","last_undegraded":"2024-03-28T01:08:27.158052+0000","last_fullsized":"2024-03-28T01:08:27.158052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.157506+0000","last_change":"2024-03-28T01:06:22.744849+0000","last_active":"2024-03-28T01:08:27.157506+0000","last_peered":"2024-03-28T01:08:27.157506+0000","last_clean":"2024-03-28T01:08:27.157506+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:08:27.157506+0000","last_undegraded":"2024-03-28T01:08:27.157506+0000","last_fullsized":"2024-03-28T01:08:27.157506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163238+0000","last_change":"2024-03-28T01:06:22.741387+0000","last_active":"2024-03-28T01:08:27.163238+0000","last_peered":"2024-03-28T01:08:27.163238+0000","last_clean":"2024-03-28T01:08:27.163238+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:08:27.163238+0000","last_undegraded":"2024-03-28T01:08:27.163238+0000","last_fullsized":"2024-03-28T01:08:27.163238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'22","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.145507+0000","last_change":"2024-03-28T01:06:22.748359+0000","last_active":"2024-03-28T01:08:27.145507+0000","last_peered":"2024-03-28T01:08:27.145507+0000","last_clean":"2024-03-28T01:08:27.145507+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:08:27.145507+0000","last_undegraded":"2024-03-28T01:08:27.145507+0000","last_fullsized":"2024-03-28T01:08:27.145507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'34","reported_seq":52,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.143883+0000","last_change":"2024-03-28T01:06:22.734562+0000","last_active":"2024-03-28T01:08:27.143883+0000","last_peered":"2024-03-28T01:08:27.143883+0000","last_clean":"2024-03-28T01:08:27.143883+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:08:27.143883+0000","last_undegraded":"2024-03-28T01:08:27.143883+0000","last_fullsized":"2024-03-28T01:08:27.143883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.135097+0000","last_change":"2024-03-28T01:06:22.739841+0000","last_active":"2024-03-28T01:08:27.135097+0000","last_peered":"2024-03-28T01:08:27.135097+0000","last_clean":"2024-03-28T01:08:27.135097+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:08:27.135097+0000","last_undegraded":"2024-03-28T01:08:27.135097+0000","last_fullsized":"2024-03-28T01:08:27.135097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.151305+0000","last_change":"2024-03-28T01:06:22.740291+0000","last_active":"2024-03-28T01:08:27.151305+0000","last_peered":"2024-03-28T01:08:27.151305+0000","last_clean":"2024-03-28T01:08:27.151305+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:08:27.151305+0000","last_undegraded":"2024-03-28T01:08:27.151305+0000","last_fullsized":"2024-03-28T01:08:27.151305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:40.818862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'20","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163392+0000","last_change":"2024-03-28T01:06:22.741844+0000","last_active":"2024-03-28T01:08:27.163392+0000","last_peered":"2024-03-28T01:08:27.163392+0000","last_clean":"2024-03-28T01:08:27.163392+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:08:27.163392+0000","last_undegraded":"2024-03-28T01:08:27.163392+0000","last_fullsized":"2024-03-28T01:08:27.163392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'16","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.161089+0000","last_change":"2024-03-28T01:06:22.722291+0000","last_active":"2024-03-28T01:08:27.161089+0000","last_peered":"2024-03-28T01:08:27.161089+0000","last_clean":"2024-03-28T01:08:27.161089+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:08:27.161089+0000","last_undegraded":"2024-03-28T01:08:27.161089+0000","last_fullsized":"2024-03-28T01:08:27.161089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'14","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.144491+0000","last_change":"2024-03-28T01:06:22.729267+0000","last_active":"2024-03-28T01:08:27.144491+0000","last_peered":"2024-03-28T01:08:27.144491+0000","last_clean":"2024-03-28T01:08:27.144491+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:08:27.144491+0000","last_undegraded":"2024-03-28T01:08:27.144491+0000","last_fullsized":"2024-03-28T01:08:27.144491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'21","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.157398+0000","last_change":"2024-03-28T01:06:22.736507+0000","last_active":"2024-03-28T01:08:27.157398+0000","last_peered":"2024-03-28T01:08:27.157398+0000","last_clean":"2024-03-28T01:08:27.157398+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:08:27.157398+0000","last_undegraded":"2024-03-28T01:08:27.157398+0000","last_fullsized":"2024-03-28T01:08:27.157398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:30:22.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'16","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.146727+0000","last_change":"2024-03-28T01:06:22.748365+0000","last_active":"2024-03-28T01:08:27.146727+0000","last_peered":"2024-03-28T01:08:27.146727+0000","last_clean":"2024-03-28T01:08:27.146727+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:08:27.146727+0000","last_undegraded":"2024-03-28T01:08:27.146727+0000","last_fullsized":"2024-03-28T01:08:27.146727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'20","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.144135+0000","last_change":"2024-03-28T01:06:22.723707+0000","last_active":"2024-03-28T01:08:27.144135+0000","last_peered":"2024-03-28T01:08:27.144135+0000","last_clean":"2024-03-28T01:08:27.144135+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:08:27.144135+0000","last_undegraded":"2024-03-28T01:08:27.144135+0000","last_fullsized":"2024-03-28T01:08:27.144135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'14","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.148941+0000","last_change":"2024-03-28T01:07:03.939021+0000","last_active":"2024-03-28T01:08:27.148941+0000","last_peered":"2024-03-28T01:08:27.148941+0000","last_clean":"2024-03-28T01:08:27.148941+0000","last_became_active":"2024-03-28T01:07:03.937565+0000","last_became_peered":"2024-03-28T01:07:03.937565+0000","last_unstale":"2024-03-28T01:08:27.148941+0000","last_undegraded":"2024-03-28T01:08:27.148941+0000","last_fullsized":"2024-03-28T01:08:27.148941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"23'16","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.148396+0000","last_change":"2024-03-28T01:06:22.731933+0000","last_active":"2024-03-28T01:08:27.148396+0000","last_peered":"2024-03-28T01:08:27.148396+0000","last_clean":"2024-03-28T01:08:27.148396+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:08:27.148396+0000","last_undegraded":"2024-03-28T01:08:27.148396+0000","last_fullsized":"2024-03-28T01:08:27.148396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:42:44.513223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'24","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.161824+0000","last_change":"2024-03-28T01:06:22.736338+0000","last_active":"2024-03-28T01:08:27.161824+0000","last_peered":"2024-03-28T01:08:27.161824+0000","last_clean":"2024-03-28T01:08:27.161824+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:08:27.161824+0000","last_undegraded":"2024-03-28T01:08:27.161824+0000","last_fullsized":"2024-03-28T01:08:27.161824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'20","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.149950+0000","last_change":"2024-03-28T01:07:03.972650+0000","last_active":"2024-03-28T01:08:27.149950+0000","last_peered":"2024-03-28T01:08:27.149950+0000","last_clean":"2024-03-28T01:08:27.149950+0000","last_became_active":"2024-03-28T01:07:03.972014+0000","last_became_peered":"2024-03-28T01:07:03.972014+0000","last_unstale":"2024-03-28T01:08:27.149950+0000","last_undegraded":"2024-03-28T01:08:27.149950+0000","last_fullsized":"2024-03-28T01:08:27.149950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"23'16","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.161696+0000","last_change":"2024-03-28T01:06:22.738077+0000","last_active":"2024-03-28T01:08:27.161696+0000","last_peered":"2024-03-28T01:08:27.161696+0000","last_clean":"2024-03-28T01:08:27.161696+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:08:27.161696+0000","last_undegraded":"2024-03-28T01:08:27.161696+0000","last_fullsized":"2024-03-28T01:08:27.161696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'16","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163743+0000","last_change":"2024-03-28T01:06:22.747627+0000","last_active":"2024-03-28T01:08:27.163743+0000","last_peered":"2024-03-28T01:08:27.163743+0000","last_clean":"2024-03-28T01:08:27.163743+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:08:27.163743+0000","last_undegraded":"2024-03-28T01:08:27.163743+0000","last_fullsized":"2024-03-28T01:08:27.163743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:05:59.892525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.769823+0000","last_change":"2024-03-28T01:06:22.739690+0000","last_active":"2024-03-28T01:08:34.769823+0000","last_peered":"2024-03-28T01:08:34.769823+0000","last_clean":"2024-03-28T01:08:34.769823+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:08:34.769823+0000","last_undegraded":"2024-03-28T01:08:34.769823+0000","last_fullsized":"2024-03-28T01:08:34.769823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'32","reported_seq":51,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.145845+0000","last_change":"2024-03-28T01:06:22.731891+0000","last_active":"2024-03-28T01:08:27.145845+0000","last_peered":"2024-03-28T01:08:27.145845+0000","last_clean":"2024-03-28T01:08:27.145845+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:08:27.145845+0000","last_undegraded":"2024-03-28T01:08:27.145845+0000","last_fullsized":"2024-03-28T01:08:27.145845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'12","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:33.247749+0000","last_change":"2024-03-28T01:06:22.740143+0000","last_active":"2024-03-28T01:08:33.247749+0000","last_peered":"2024-03-28T01:08:33.247749+0000","last_clean":"2024-03-28T01:08:33.247749+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:08:33.247749+0000","last_undegraded":"2024-03-28T01:08:33.247749+0000","last_fullsized":"2024-03-28T01:08:33.247749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.769719+0000","last_change":"2024-03-28T01:06:22.735315+0000","last_active":"2024-03-28T01:08:34.769719+0000","last_peered":"2024-03-28T01:08:34.769719+0000","last_clean":"2024-03-28T01:08:34.769719+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:08:34.769719+0000","last_undegraded":"2024-03-28T01:08:34.769719+0000","last_fullsized":"2024-03-28T01:08:34.769719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'14","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.457154+0000","last_change":"2024-03-28T01:07:03.942155+0000","last_active":"2024-03-28T01:08:30.457154+0000","last_peered":"2024-03-28T01:08:30.457154+0000","last_clean":"2024-03-28T01:08:30.457154+0000","last_became_active":"2024-03-28T01:07:03.941729+0000","last_became_peered":"2024-03-28T01:07:03.941729+0000","last_unstale":"2024-03-28T01:08:30.457154+0000","last_undegraded":"2024-03-28T01:08:30.457154+0000","last_fullsized":"2024-03-28T01:08:30.457154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'16","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.174133+0000","last_change":"2024-03-28T01:07:03.930604+0000","last_active":"2024-03-28T01:08:27.174133+0000","last_peered":"2024-03-28T01:08:27.174133+0000","last_clean":"2024-03-28T01:08:27.174133+0000","last_became_active":"2024-03-28T01:07:03.929334+0000","last_became_peered":"2024-03-28T01:07:03.929334+0000","last_unstale":"2024-03-28T01:08:27.174133+0000","last_undegraded":"2024-03-28T01:08:27.174133+0000","last_fullsized":"2024-03-28T01:08:27.174133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"23'24","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.453173+0000","last_change":"2024-03-28T01:06:22.733703+0000","last_active":"2024-03-28T01:08:30.453173+0000","last_peered":"2024-03-28T01:08:30.453173+0000","last_clean":"2024-03-28T01:08:30.453173+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:08:30.453173+0000","last_undegraded":"2024-03-28T01:08:30.453173+0000","last_fullsized":"2024-03-28T01:08:30.453173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.158253+0000","last_change":"2024-03-28T01:06:22.730832+0000","last_active":"2024-03-28T01:08:27.158253+0000","last_peered":"2024-03-28T01:08:27.158253+0000","last_clean":"2024-03-28T01:08:27.158253+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:08:27.158253+0000","last_undegraded":"2024-03-28T01:08:27.158253+0000","last_fullsized":"2024-03-28T01:08:27.158253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:05.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'14","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.149951+0000","last_change":"2024-03-28T01:06:22.723843+0000","last_active":"2024-03-28T01:08:27.149951+0000","last_peered":"2024-03-28T01:08:27.149951+0000","last_clean":"2024-03-28T01:08:27.149951+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:08:27.149951+0000","last_undegraded":"2024-03-28T01:08:27.149951+0000","last_fullsized":"2024-03-28T01:08:27.149951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:33.230806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.141112+0000","last_change":"2024-03-28T01:06:21.728329+0000","last_active":"2024-03-28T01:08:19.141112+0000","last_peered":"2024-03-28T01:08:19.141112+0000","last_clean":"2024-03-28T01:08:19.141112+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:08:19.141112+0000","last_undegraded":"2024-03-28T01:08:19.141112+0000","last_fullsized":"2024-03-28T01:08:19.141112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'16","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.184439+0000","last_change":"2024-03-28T01:08:35.184493+0000","last_active":"2024-03-28T01:08:35.184439+0000","last_peered":"2024-03-28T01:08:35.184439+0000","last_clean":"2024-03-28T01:08:35.184439+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:08:35.184439+0000","last_undegraded":"2024-03-28T01:08:35.184439+0000","last_fullsized":"2024-03-28T01:08:35.184439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:35.184290+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:35.184290+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.184290+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:45:27.934451+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145219+0000","last_change":"2024-03-28T01:06:21.733590+0000","last_active":"2024-03-28T01:08:19.145219+0000","last_peered":"2024-03-28T01:08:19.145219+0000","last_clean":"2024-03-28T01:08:19.145219+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:08:19.145219+0000","last_undegraded":"2024-03-28T01:08:19.145219+0000","last_fullsized":"2024-03-28T01:08:19.145219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.890987+0000","last_change":"2024-03-28T01:08:34.891029+0000","last_active":"2024-03-28T01:08:34.890987+0000","last_peered":"2024-03-28T01:08:34.890987+0000","last_clean":"2024-03-28T01:08:34.890987+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:08:34.890987+0000","last_undegraded":"2024-03-28T01:08:34.890987+0000","last_fullsized":"2024-03-28T01:08:34.890987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:34.890891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:34.890891+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.890891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:53:19.453887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'16","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163058+0000","last_change":"2024-03-28T01:06:22.731850+0000","last_active":"2024-03-28T01:08:27.163058+0000","last_peered":"2024-03-28T01:08:27.163058+0000","last_clean":"2024-03-28T01:08:27.163058+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:08:27.163058+0000","last_undegraded":"2024-03-28T01:08:27.163058+0000","last_fullsized":"2024-03-28T01:08:27.163058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580460+0000","last_change":"2024-03-28T01:06:21.734157+0000","last_active":"2024-03-28T01:08:19.580460+0000","last_peered":"2024-03-28T01:08:19.580460+0000","last_clean":"2024-03-28T01:08:19.580460+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:08:19.580460+0000","last_undegraded":"2024-03-28T01:08:19.580460+0000","last_fullsized":"2024-03-28T01:08:19.580460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'24","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.164360+0000","last_change":"2024-03-28T01:06:22.724994+0000","last_active":"2024-03-28T01:08:27.164360+0000","last_peered":"2024-03-28T01:08:27.164360+0000","last_clean":"2024-03-28T01:08:27.164360+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:08:27.164360+0000","last_undegraded":"2024-03-28T01:08:27.164360+0000","last_fullsized":"2024-03-28T01:08:27.164360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145869+0000","last_change":"2024-03-28T01:06:21.732327+0000","last_active":"2024-03-28T01:08:19.145869+0000","last_peered":"2024-03-28T01:08:19.145869+0000","last_clean":"2024-03-28T01:08:19.145869+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:08:19.145869+0000","last_undegraded":"2024-03-28T01:08:19.145869+0000","last_fullsized":"2024-03-28T01:08:19.145869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'20","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.149158+0000","last_change":"2024-03-28T01:06:22.748218+0000","last_active":"2024-03-28T01:08:27.149158+0000","last_peered":"2024-03-28T01:08:27.149158+0000","last_clean":"2024-03-28T01:08:27.149158+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:08:27.149158+0000","last_undegraded":"2024-03-28T01:08:27.149158+0000","last_fullsized":"2024-03-28T01:08:27.149158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582176+0000","last_change":"2024-03-28T01:06:21.730156+0000","last_active":"2024-03-28T01:08:19.582176+0000","last_peered":"2024-03-28T01:08:19.582176+0000","last_clean":"2024-03-28T01:08:19.582176+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:08:19.582176+0000","last_undegraded":"2024-03-28T01:08:19.582176+0000","last_fullsized":"2024-03-28T01:08:19.582176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'16","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.143527+0000","last_change":"2024-03-28T01:07:03.970890+0000","last_active":"2024-03-28T01:08:27.143527+0000","last_peered":"2024-03-28T01:08:27.143527+0000","last_clean":"2024-03-28T01:08:27.143527+0000","last_became_active":"2024-03-28T01:07:03.970312+0000","last_became_peered":"2024-03-28T01:07:03.970312+0000","last_unstale":"2024-03-28T01:08:27.143527+0000","last_undegraded":"2024-03-28T01:08:27.143527+0000","last_fullsized":"2024-03-28T01:08:27.143527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.895342+0000","last_change":"2024-03-28T01:06:21.724797+0000","last_active":"2024-03-28T01:08:24.895342+0000","last_peered":"2024-03-28T01:08:24.895342+0000","last_clean":"2024-03-28T01:08:24.895342+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:08:24.895342+0000","last_undegraded":"2024-03-28T01:08:24.895342+0000","last_fullsized":"2024-03-28T01:08:24.895342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'22","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.150264+0000","last_change":"2024-03-28T01:06:22.738292+0000","last_active":"2024-03-28T01:08:27.150264+0000","last_peered":"2024-03-28T01:08:27.150264+0000","last_clean":"2024-03-28T01:08:27.150264+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:08:27.150264+0000","last_undegraded":"2024-03-28T01:08:27.150264+0000","last_fullsized":"2024-03-28T01:08:27.150264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:22.083276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'121","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:32.088565+0000","last_change":"2024-03-28T01:06:21.722965+0000","last_active":"2024-03-28T01:08:32.088565+0000","last_peered":"2024-03-28T01:08:32.088565+0000","last_clean":"2024-03-28T01:08:32.088565+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:08:32.088565+0000","last_undegraded":"2024-03-28T01:08:32.088565+0000","last_fullsized":"2024-03-28T01:08:32.088565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148885,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'13","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.148582+0000","last_change":"2024-03-28T01:06:22.742359+0000","last_active":"2024-03-28T01:08:27.148582+0000","last_peered":"2024-03-28T01:08:27.148582+0000","last_clean":"2024-03-28T01:08:27.148582+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:08:27.148582+0000","last_undegraded":"2024-03-28T01:08:27.148582+0000","last_fullsized":"2024-03-28T01:08:27.148582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:12:39.922842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.230553+0000","last_change":"2024-03-28T01:08:34.230625+0000","last_active":"2024-03-28T01:08:34.230553+0000","last_peered":"2024-03-28T01:08:34.230553+0000","last_clean":"2024-03-28T01:08:34.230553+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:08:34.230553+0000","last_undegraded":"2024-03-28T01:08:34.230553+0000","last_fullsized":"2024-03-28T01:08:34.230553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-03-28T01:08:34.230383+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-03-28T01:08:34.230383+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.230383+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:59:00.767286+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'19","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.159844+0000","last_change":"2024-03-28T01:06:22.736705+0000","last_active":"2024-03-28T01:08:27.159844+0000","last_peered":"2024-03-28T01:08:27.159844+0000","last_clean":"2024-03-28T01:08:27.159844+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:08:27.159844+0000","last_undegraded":"2024-03-28T01:08:27.159844+0000","last_fullsized":"2024-03-28T01:08:27.159844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580799+0000","last_change":"2024-03-28T01:06:21.729725+0000","last_active":"2024-03-28T01:08:19.580799+0000","last_peered":"2024-03-28T01:08:19.580799+0000","last_clean":"2024-03-28T01:08:19.580799+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:08:19.580799+0000","last_undegraded":"2024-03-28T01:08:19.580799+0000","last_fullsized":"2024-03-28T01:08:19.580799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'30","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.150929+0000","last_change":"2024-03-28T01:06:22.742197+0000","last_active":"2024-03-28T01:08:27.150929+0000","last_peered":"2024-03-28T01:08:27.150929+0000","last_clean":"2024-03-28T01:08:27.150929+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:08:27.150929+0000","last_undegraded":"2024-03-28T01:08:27.150929+0000","last_fullsized":"2024-03-28T01:08:27.150929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.145073+0000","last_change":"2024-03-28T01:06:21.730264+0000","last_active":"2024-03-28T01:08:19.145073+0000","last_peered":"2024-03-28T01:08:19.145073+0000","last_clean":"2024-03-28T01:08:19.145073+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:08:19.145073+0000","last_undegraded":"2024-03-28T01:08:19.145073+0000","last_fullsized":"2024-03-28T01:08:19.145073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:39:24.046068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'14","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.153466+0000","last_change":"2024-03-28T01:06:22.719323+0000","last_active":"2024-03-28T01:08:27.153466+0000","last_peered":"2024-03-28T01:08:27.153466+0000","last_clean":"2024-03-28T01:08:27.153466+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:08:27.153466+0000","last_undegraded":"2024-03-28T01:08:27.153466+0000","last_fullsized":"2024-03-28T01:08:27.153466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:51:45.489649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'12","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.404429+0000","last_change":"2024-03-28T01:08:34.404464+0000","last_active":"2024-03-28T01:08:34.404429+0000","last_peered":"2024-03-28T01:08:34.404429+0000","last_clean":"2024-03-28T01:08:34.404429+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:08:34.404429+0000","last_undegraded":"2024-03-28T01:08:34.404429+0000","last_fullsized":"2024-03-28T01:08:34.404429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:34.404344+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:34.404344+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.404344+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:15:06.611896+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'12","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.146405+0000","last_change":"2024-03-28T01:06:22.733485+0000","last_active":"2024-03-28T01:08:27.146405+0000","last_peered":"2024-03-28T01:08:27.146405+0000","last_clean":"2024-03-28T01:08:27.146405+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:08:27.146405+0000","last_undegraded":"2024-03-28T01:08:27.146405+0000","last_fullsized":"2024-03-28T01:08:27.146405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:38:02.317476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.142534+0000","last_change":"2024-03-28T01:06:21.722118+0000","last_active":"2024-03-28T01:08:19.142534+0000","last_peered":"2024-03-28T01:08:19.142534+0000","last_clean":"2024-03-28T01:08:19.142534+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:08:19.142534+0000","last_undegraded":"2024-03-28T01:08:19.142534+0000","last_fullsized":"2024-03-28T01:08:19.142534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:15:33.242860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'10","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.144450+0000","last_change":"2024-03-28T01:06:22.745634+0000","last_active":"2024-03-28T01:08:27.144450+0000","last_peered":"2024-03-28T01:08:27.144450+0000","last_clean":"2024-03-28T01:08:27.144450+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:08:27.144450+0000","last_undegraded":"2024-03-28T01:08:27.144450+0000","last_fullsized":"2024-03-28T01:08:27.144450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:30:56.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580941+0000","last_change":"2024-03-28T01:06:21.722501+0000","last_active":"2024-03-28T01:08:19.580941+0000","last_peered":"2024-03-28T01:08:19.580941+0000","last_clean":"2024-03-28T01:08:19.580941+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:08:19.580941+0000","last_undegraded":"2024-03-28T01:08:19.580941+0000","last_fullsized":"2024-03-28T01:08:19.580941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'18","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.142565+0000","last_change":"2024-03-28T01:06:22.750703+0000","last_active":"2024-03-28T01:08:27.142565+0000","last_peered":"2024-03-28T01:08:27.142565+0000","last_clean":"2024-03-28T01:08:27.142565+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:08:27.142565+0000","last_undegraded":"2024-03-28T01:08:27.142565+0000","last_fullsized":"2024-03-28T01:08:27.142565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:12.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.057841+0000","last_change":"2024-03-28T01:08:35.057893+0000","last_active":"2024-03-28T01:08:35.057841+0000","last_peered":"2024-03-28T01:08:35.057841+0000","last_clean":"2024-03-28T01:08:35.057841+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:08:35.057841+0000","last_undegraded":"2024-03-28T01:08:35.057841+0000","last_fullsized":"2024-03-28T01:08:35.057841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-03-28T01:08:35.057718+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-03-28T01:08:35.057718+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.057718+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:53:19.620715+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'14","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163449+0000","last_change":"2024-03-28T01:06:22.751125+0000","last_active":"2024-03-28T01:08:27.163449+0000","last_peered":"2024-03-28T01:08:27.163449+0000","last_clean":"2024-03-28T01:08:27.163449+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:08:27.163449+0000","last_undegraded":"2024-03-28T01:08:27.163449+0000","last_fullsized":"2024-03-28T01:08:27.163449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:41:37.850626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580761+0000","last_change":"2024-03-28T01:06:21.733230+0000","last_active":"2024-03-28T01:08:19.580761+0000","last_peered":"2024-03-28T01:08:19.580761+0000","last_clean":"2024-03-28T01:08:19.580761+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:08:19.580761+0000","last_undegraded":"2024-03-28T01:08:19.580761+0000","last_fullsized":"2024-03-28T01:08:19.580761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'14","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.147355+0000","last_change":"2024-03-28T01:06:22.748267+0000","last_active":"2024-03-28T01:08:27.147355+0000","last_peered":"2024-03-28T01:08:27.147355+0000","last_clean":"2024-03-28T01:08:27.147355+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:08:27.147355+0000","last_undegraded":"2024-03-28T01:08:27.147355+0000","last_fullsized":"2024-03-28T01:08:27.147355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580583+0000","last_change":"2024-03-28T01:06:11.770808+0000","last_active":"2024-03-28T01:08:19.580583+0000","last_peered":"2024-03-28T01:08:19.580583+0000","last_clean":"2024-03-28T01:08:19.580583+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:08:19.580583+0000","last_undegraded":"2024-03-28T01:08:19.580583+0000","last_fullsized":"2024-03-28T01:08:19.580583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:10.708388+0000","last_clean_scrub_stamp":"2024-03-28T01:06:10.708388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:33:28.106942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580876+0000","last_change":"2024-03-28T01:06:21.724104+0000","last_active":"2024-03-28T01:08:19.580876+0000","last_peered":"2024-03-28T01:08:19.580876+0000","last_clean":"2024-03-28T01:08:19.580876+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:08:19.580876+0000","last_undegraded":"2024-03-28T01:08:19.580876+0000","last_fullsized":"2024-03-28T01:08:19.580876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'16","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.454080+0000","last_change":"2024-03-28T01:06:22.739184+0000","last_active":"2024-03-28T01:08:30.454080+0000","last_peered":"2024-03-28T01:08:30.454080+0000","last_clean":"2024-03-28T01:08:30.454080+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:08:30.454080+0000","last_undegraded":"2024-03-28T01:08:30.454080+0000","last_fullsized":"2024-03-28T01:08:30.454080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580554+0000","last_change":"2024-03-28T01:06:21.736576+0000","last_active":"2024-03-28T01:08:19.580554+0000","last_peered":"2024-03-28T01:08:19.580554+0000","last_clean":"2024-03-28T01:08:19.580554+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:08:19.580554+0000","last_undegraded":"2024-03-28T01:08:19.580554+0000","last_fullsized":"2024-03-28T01:08:19.580554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.457109+0000","last_change":"2024-03-28T01:06:22.729635+0000","last_active":"2024-03-28T01:08:30.457109+0000","last_peered":"2024-03-28T01:08:30.457109+0000","last_clean":"2024-03-28T01:08:30.457109+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:08:30.457109+0000","last_undegraded":"2024-03-28T01:08:30.457109+0000","last_fullsized":"2024-03-28T01:08:30.457109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'10","reported_seq":49,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.387134+0000","last_change":"2024-03-28T01:08:35.387179+0000","last_active":"2024-03-28T01:08:35.387134+0000","last_peered":"2024-03-28T01:08:35.387134+0000","last_clean":"2024-03-28T01:08:35.387134+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:08:35.387134+0000","last_undegraded":"2024-03-28T01:08:35.387134+0000","last_fullsized":"2024-03-28T01:08:35.387134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:35.387008+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:35.387008+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.387008+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:12:14.987632+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'17","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.149884+0000","last_change":"2024-03-28T01:06:22.720175+0000","last_active":"2024-03-28T01:08:27.149884+0000","last_peered":"2024-03-28T01:08:27.149884+0000","last_clean":"2024-03-28T01:08:27.149884+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:08:27.149884+0000","last_undegraded":"2024-03-28T01:08:27.149884+0000","last_fullsized":"2024-03-28T01:08:27.149884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:24.891223+0000","last_change":"2024-03-28T01:06:21.724947+0000","last_active":"2024-03-28T01:08:24.891223+0000","last_peered":"2024-03-28T01:08:24.891223+0000","last_clean":"2024-03-28T01:08:24.891223+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:08:24.891223+0000","last_undegraded":"2024-03-28T01:08:24.891223+0000","last_fullsized":"2024-03-28T01:08:24.891223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'14","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.153217+0000","last_change":"2024-03-28T01:06:22.721839+0000","last_active":"2024-03-28T01:08:27.153217+0000","last_peered":"2024-03-28T01:08:27.153217+0000","last_clean":"2024-03-28T01:08:27.153217+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:08:27.153217+0000","last_undegraded":"2024-03-28T01:08:27.153217+0000","last_fullsized":"2024-03-28T01:08:27.153217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:45.436686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.460110+0000","last_change":"2024-03-28T01:06:21.718177+0000","last_active":"2024-03-28T01:08:30.460110+0000","last_peered":"2024-03-28T01:08:30.460110+0000","last_clean":"2024-03-28T01:08:30.460110+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:08:30.460110+0000","last_undegraded":"2024-03-28T01:08:30.460110+0000","last_fullsized":"2024-03-28T01:08:30.460110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'10","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163608+0000","last_change":"2024-03-28T01:06:22.745379+0000","last_active":"2024-03-28T01:08:27.163608+0000","last_peered":"2024-03-28T01:08:27.163608+0000","last_clean":"2024-03-28T01:08:27.163608+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:08:27.163608+0000","last_undegraded":"2024-03-28T01:08:27.163608+0000","last_fullsized":"2024-03-28T01:08:27.163608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:12.051336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582559+0000","last_change":"2024-03-28T01:06:21.735005+0000","last_active":"2024-03-28T01:08:19.582559+0000","last_peered":"2024-03-28T01:08:19.582559+0000","last_clean":"2024-03-28T01:08:19.582559+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:08:19.582559+0000","last_undegraded":"2024-03-28T01:08:19.582559+0000","last_fullsized":"2024-03-28T01:08:19.582559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:45:21.069690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'18","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.460049+0000","last_change":"2024-03-28T01:06:22.724230+0000","last_active":"2024-03-28T01:08:30.460049+0000","last_peered":"2024-03-28T01:08:30.460049+0000","last_clean":"2024-03-28T01:08:30.460049+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:08:30.460049+0000","last_undegraded":"2024-03-28T01:08:30.460049+0000","last_fullsized":"2024-03-28T01:08:30.460049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'6","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:33.247233+0000","last_change":"2024-03-28T01:06:21.725001+0000","last_active":"2024-03-28T01:08:33.247233+0000","last_peered":"2024-03-28T01:08:33.247233+0000","last_clean":"2024-03-28T01:08:33.247233+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:08:33.247233+0000","last_undegraded":"2024-03-28T01:08:33.247233+0000","last_fullsized":"2024-03-28T01:08:33.247233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'13","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.769497+0000","last_change":"2024-03-28T01:06:22.748406+0000","last_active":"2024-03-28T01:08:34.769497+0000","last_peered":"2024-03-28T01:08:34.769497+0000","last_clean":"2024-03-28T01:08:34.769497+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:08:34.769497+0000","last_undegraded":"2024-03-28T01:08:34.769497+0000","last_fullsized":"2024-03-28T01:08:34.769497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.146118+0000","last_change":"2024-03-28T01:06:21.737254+0000","last_active":"2024-03-28T01:08:19.146118+0000","last_peered":"2024-03-28T01:08:19.146118+0000","last_clean":"2024-03-28T01:08:19.146118+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:08:19.146118+0000","last_undegraded":"2024-03-28T01:08:19.146118+0000","last_fullsized":"2024-03-28T01:08:19.146118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:07.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'14","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:32.500854+0000","last_change":"2024-03-28T01:06:22.725643+0000","last_active":"2024-03-28T01:08:32.500854+0000","last_peered":"2024-03-28T01:08:32.500854+0000","last_clean":"2024-03-28T01:08:32.500854+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:08:32.500854+0000","last_undegraded":"2024-03-28T01:08:32.500854+0000","last_fullsized":"2024-03-28T01:08:32.500854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.580249+0000","last_change":"2024-03-28T01:06:21.732849+0000","last_active":"2024-03-28T01:08:19.580249+0000","last_peered":"2024-03-28T01:08:19.580249+0000","last_clean":"2024-03-28T01:08:19.580249+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:08:19.580249+0000","last_undegraded":"2024-03-28T01:08:19.580249+0000","last_fullsized":"2024-03-28T01:08:19.580249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:39:58.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.135941+0000","last_change":"2024-03-28T01:06:22.718047+0000","last_active":"2024-03-28T01:08:27.135941+0000","last_peered":"2024-03-28T01:08:27.135941+0000","last_clean":"2024-03-28T01:08:27.135941+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:08:27.135941+0000","last_undegraded":"2024-03-28T01:08:27.135941+0000","last_fullsized":"2024-03-28T01:08:27.135941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:18:09.004654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:33.247375+0000","last_change":"2024-03-28T01:06:21.728280+0000","last_active":"2024-03-28T01:08:33.247375+0000","last_peered":"2024-03-28T01:08:33.247375+0000","last_clean":"2024-03-28T01:08:33.247375+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:08:33.247375+0000","last_undegraded":"2024-03-28T01:08:33.247375+0000","last_fullsized":"2024-03-28T01:08:33.247375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'18","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.769624+0000","last_change":"2024-03-28T01:06:22.724126+0000","last_active":"2024-03-28T01:08:34.769624+0000","last_peered":"2024-03-28T01:08:34.769624+0000","last_clean":"2024-03-28T01:08:34.769624+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:08:34.769624+0000","last_undegraded":"2024-03-28T01:08:34.769624+0000","last_fullsized":"2024-03-28T01:08:34.769624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.005454+0000","last_change":"2024-03-28T01:08:34.005454+0000","last_active":"2024-03-28T01:08:34.005454+0000","last_peered":"2024-03-28T01:08:34.005454+0000","last_clean":"2024-03-28T01:08:34.005454+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:08:34.005454+0000","last_undegraded":"2024-03-28T01:08:34.005454+0000","last_fullsized":"2024-03-28T01:08:34.005454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:34.005348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:34.005348+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.005348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:18:05.676086+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'16","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.163123+0000","last_change":"2024-03-28T01:06:22.741641+0000","last_active":"2024-03-28T01:08:27.163123+0000","last_peered":"2024-03-28T01:08:27.163123+0000","last_clean":"2024-03-28T01:08:27.163123+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:08:27.163123+0000","last_undegraded":"2024-03-28T01:08:27.163123+0000","last_fullsized":"2024-03-28T01:08:27.163123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582048+0000","last_change":"2024-03-28T01:06:21.735308+0000","last_active":"2024-03-28T01:08:19.582048+0000","last_peered":"2024-03-28T01:08:19.582048+0000","last_clean":"2024-03-28T01:08:19.582048+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:08:19.582048+0000","last_undegraded":"2024-03-28T01:08:19.582048+0000","last_fullsized":"2024-03-28T01:08:19.582048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:49:44.423100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'10","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.150034+0000","last_change":"2024-03-28T01:06:22.732366+0000","last_active":"2024-03-28T01:08:27.150034+0000","last_peered":"2024-03-28T01:08:27.150034+0000","last_clean":"2024-03-28T01:08:27.150034+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:08:27.150034+0000","last_undegraded":"2024-03-28T01:08:27.150034+0000","last_fullsized":"2024-03-28T01:08:27.150034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:58:12.405402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'29","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.150636+0000","last_change":"2024-03-28T01:06:22.737862+0000","last_active":"2024-03-28T01:08:27.150636+0000","last_peered":"2024-03-28T01:08:27.150636+0000","last_clean":"2024-03-28T01:08:27.150636+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:08:27.150636+0000","last_undegraded":"2024-03-28T01:08:27.150636+0000","last_fullsized":"2024-03-28T01:08:27.150636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:58:14.382542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.581500+0000","last_change":"2024-03-28T01:06:21.735004+0000","last_active":"2024-03-28T01:08:19.581500+0000","last_peered":"2024-03-28T01:08:19.581500+0000","last_clean":"2024-03-28T01:08:19.581500+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:08:19.581500+0000","last_undegraded":"2024-03-28T01:08:19.581500+0000","last_fullsized":"2024-03-28T01:08:19.581500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:33:32.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'18","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.151227+0000","last_change":"2024-03-28T01:06:22.732987+0000","last_active":"2024-03-28T01:08:27.151227+0000","last_peered":"2024-03-28T01:08:27.151227+0000","last_clean":"2024-03-28T01:08:27.151227+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:08:27.151227+0000","last_undegraded":"2024-03-28T01:08:27.151227+0000","last_fullsized":"2024-03-28T01:08:27.151227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:44:38.748726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:19.582016+0000","last_change":"2024-03-28T01:06:21.731143+0000","last_active":"2024-03-28T01:08:19.582016+0000","last_peered":"2024-03-28T01:08:19.582016+0000","last_clean":"2024-03-28T01:08:19.582016+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:08:19.582016+0000","last_undegraded":"2024-03-28T01:08:19.582016+0000","last_fullsized":"2024-03-28T01:08:19.582016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:14:39.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'12","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:30.457207+0000","last_change":"2024-03-28T01:07:03.942345+0000","last_active":"2024-03-28T01:08:30.457207+0000","last_peered":"2024-03-28T01:08:30.457207+0000","last_clean":"2024-03-28T01:08:30.457207+0000","last_became_active":"2024-03-28T01:07:03.941880+0000","last_became_peered":"2024-03-28T01:07:03.941880+0000","last_unstale":"2024-03-28T01:08:30.457207+0000","last_undegraded":"2024-03-28T01:08:30.457207+0000","last_fullsized":"2024-03-28T01:08:30.457207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:25.453174+0000","last_change":"2024-03-28T01:06:21.723032+0000","last_active":"2024-03-28T01:08:25.453174+0000","last_peered":"2024-03-28T01:08:25.453174+0000","last_clean":"2024-03-28T01:08:25.453174+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:08:25.453174+0000","last_undegraded":"2024-03-28T01:08:25.453174+0000","last_fullsized":"2024-03-28T01:08:25.453174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:20.659810+0000","last_clean_scrub_stamp":"2024-03-28T01:06:20.659810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'10","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:27.159742+0000","last_change":"2024-03-28T01:06:22.739585+0000","last_active":"2024-03-28T01:08:27.159742+0000","last_peered":"2024-03-28T01:08:27.159742+0000","last_clean":"2024-03-28T01:08:27.159742+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:08:27.159742+0000","last_undegraded":"2024-03-28T01:08:27.159742+0000","last_fullsized":"2024-03-28T01:08:27.159742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:06:21.673395+0000","last_clean_scrub_stamp":"2024-03-28T01:06:21.673395+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:33.245717+0000","last_change":"2024-03-28T01:08:33.245788+0000","last_active":"2024-03-28T01:08:33.245717+0000","last_peered":"2024-03-28T01:08:33.245717+0000","last_clean":"2024-03-28T01:08:33.245717+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:08:33.245717+0000","last_undegraded":"2024-03-28T01:08:33.245717+0000","last_fullsized":"2024-03-28T01:08:33.245717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-03-28T01:08:33.245481+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-03-28T01:08:33.245481+0000","last_clean_scrub_stamp":"2024-03-28T01:08:33.245481+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:56:41.786549+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":21,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":42,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1028,"ondisk_log_size":1028,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":150794,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182,"num_write_kb":239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":301588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":190,"ondisk_log_size":190,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":856064,"data_stored":680429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":401408,"data_stored":238352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":89563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":89596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":421888,"data_stored":239475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":90584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":89627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":851968,"data_stored":680476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":149996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":148885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":6,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":6,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:08:38.372 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.373 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.373 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.373 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.373 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.373 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.373 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.374 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.374 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.374 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.374 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.374 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.374 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.375 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.375 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.375 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.375 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.375 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.375 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.375 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.376 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.376 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.376 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.377 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.377 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.377 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.377 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.377 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.377 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.377 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.378 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.378 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.378 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.378 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.378 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.378 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.379 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.379 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.379 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.379 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.379 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.379 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.380 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.380 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.380 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.380 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.380 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.380 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.381 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.381 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.381 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.381 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.381 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.381 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.382 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.382 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.382 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.382 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.382 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.383 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-28T01:06:10.708388+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=10, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.383 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.383 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.383 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.383 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.383 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.384 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.384 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.384 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.384 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.384 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.384 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.384 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.385 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.385 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.385 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.385 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.385 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.385 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.386 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.386 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.386 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.386 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.386 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.386 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.386 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.387 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.387 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.387 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-03-28T01:06:20.659810+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.387 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-28T01:06:21.673395+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=6, tm_sec=21, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=1, tm_min=8, tm_sec=30, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T01:08:38.387 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-28T01:08:44.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:08:44.880 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:08:58.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T01:08:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.633+0000 7fcdb2800640 1 Processor -- start 2024-03-28T01:08:58.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.633+0000 7fcdb2800640 1 -- start start 2024-03-28T01:08:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.635+0000 7fcdb2800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcdac13f500 0x7fcdac1485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.635+0000 7fcdb2800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac148f70 con 0x7fcdac0b2130 2024-03-28T01:08:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.635+0000 7fcdb2800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0b2510 con 0x7fcdac148b20 2024-03-28T01:08:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.635+0000 7fcdb2800640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac0b4670 con 0x7fcdac13f500 2024-03-28T01:08:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.636+0000 7fcdb1400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcdac0b2130 0x7fcdac13edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37338/0 (socket says 172.21.15.33:37338) 2024-03-28T01:08:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.636+0000 7fcdb0a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcdac13f500 0x7fcdac1485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.636+0000 7fcdb1400640 1 -- 172.21.15.33:0/3623709511 learned_addr learned my addr 172.21.15.33:0/3623709511 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.636+0000 7fcdb0a00640 1 -- 172.21.15.33:0/3623709511 >> v1:172.21.15.194:6789/0 conn(0x7fcdac148b20 legacy=0x7fcdac14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:58.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.636+0000 7fcdb0a00640 1 -- 172.21.15.33:0/3623709511 >> v1:172.21.15.33:6789/0 conn(0x7fcdac0b2130 legacy=0x7fcdac13edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:58.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.636+0000 7fcdb0a00640 1 -- 172.21.15.33:0/3623709511 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac06bef0 con 0x7fcdac13f500 2024-03-28T01:08:58.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.637+0000 7fcdb0a00640 1 --2- 172.21.15.33:0/3623709511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1485e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097a0 tx=0x7fcd9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e71c83c46682b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:58.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.638+0000 7fcdabe00640 1 -- 172.21.15.33:0/3623709511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980315a0 con 0x7fcdac13f500 2024-03-28T01:08:58.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.638+0000 7fcdabe00640 1 -- 172.21.15.33:0/3623709511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98037070 con 0x7fcdac13f500 2024-03-28T01:08:58.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.638+0000 7fcdabe00640 1 -- 172.21.15.33:0/3623709511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98031960 con 0x7fcdac13f500 2024-03-28T01:08:58.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.639+0000 7fcdb2800640 1 -- 172.21.15.33:0/3623709511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 msgr2=0x7fcdac1485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:58.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.639+0000 7fcdb2800640 1 --2- 172.21.15.33:0/3623709511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1485e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097a0 tx=0x7fcd9802b840 comp rx=0 tx=0).stop 2024-03-28T01:08:58.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.639+0000 7fcdb2800640 1 -- 172.21.15.33:0/3623709511 shutdown_connections 2024-03-28T01:08:58.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.639+0000 7fcdb2800640 1 --2- 172.21.15.33:0/3623709511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1485e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.639+0000 7fcdb2800640 1 -- 172.21.15.33:0/3623709511 >> 172.21.15.33:0/3623709511 conn(0x7fcdac0b89c0 msgr2=0x7fcdac0b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:58.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.639+0000 7fcdb2800640 1 -- 172.21.15.33:0/3623709511 shutdown_connections 2024-03-28T01:08:58.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.639+0000 7fcdb2800640 1 -- 172.21.15.33:0/3623709511 wait complete. 2024-03-28T01:08:58.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.640+0000 7fcdb2800640 1 Processor -- start 2024-03-28T01:08:58.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.640+0000 7fcdb2800640 1 -- start start 2024-03-28T01:08:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.641+0000 7fcdb2800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcdac0b2130 0x7fcdac1458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.641+0000 7fcdb2800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.641+0000 7fcdb2800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdac148b20 0x7fcdac1dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.641+0000 7fcdb2800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0b4ec0 con 0x7fcdac13f500 2024-03-28T01:08:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.641+0000 7fcdb2800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac06b6d0 con 0x7fcdac148b20 2024-03-28T01:08:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.641+0000 7fcdb2800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac072d20 con 0x7fcdac0b2130 2024-03-28T01:08:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.641+0000 7fcdb0a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36180/0 (socket says 172.21.15.33:36180) 2024-03-28T01:08:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 -- 172.21.15.33:0/2719318760 learned_addr learned my addr 172.21.15.33:0/2719318760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb1400640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcdac0b2130 0x7fcdac1458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb1e00640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdac148b20 0x7fcdac1dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 -- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcdac0b2130 msgr2=0x7fcdac1458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcdac0b2130 0x7fcdac1458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 -- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdac148b20 msgr2=0x7fcdac1dc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdac148b20 0x7fcdac1dc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 -- 172.21.15.33:0/2719318760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac144e00 con 0x7fcdac13f500 2024-03-28T01:08:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb1400640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcdac0b2130 0x7fcdac1458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.642+0000 7fcdb0a00640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1448c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd980098c0 tx=0x7fcd980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.643+0000 7fcda9600640 1 -- 172.21.15.33:0/2719318760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98032840 con 0x7fcdac13f500 2024-03-28T01:08:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.643+0000 7fcda9600640 1 -- 172.21.15.33:0/2719318760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd980329e0 con 0x7fcdac13f500 2024-03-28T01:08:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.643+0000 7fcda9600640 1 -- 172.21.15.33:0/2719318760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98032dd0 con 0x7fcdac13f500 2024-03-28T01:08:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.643+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac145100 con 0x7fcdac13f500 2024-03-28T01:08:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.643+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac0af640 con 0x7fcdac13f500 2024-03-28T01:08:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.645+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac075310 con 0x7fcdac13f500 2024-03-28T01:08:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.645+0000 7fcda9600640 1 -- 172.21.15.33:0/2719318760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcd9803b3f0 con 0x7fcdac13f500 2024-03-28T01:08:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.646+0000 7fcda9600640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd8c06bc00 0x7fcd8c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.646+0000 7fcda9600640 1 -- 172.21.15.33:0/2719318760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fcd9802f030 con 0x7fcdac13f500 2024-03-28T01:08:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.647+0000 7fcdb1400640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd8c06bc00 0x7fcd8c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.647+0000 7fcdb1400640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd8c06bc00 0x7fcd8c06e0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcda0005880 tx=0x7fcda0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:58.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.651+0000 7fcda9600640 1 -- 172.21.15.33:0/2719318760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fcd98040070 con 0x7fcdac13f500 2024-03-28T01:08:58.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.873+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 --> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdac06b470 con 0x7fcd8c06bc00 2024-03-28T01:08:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.883+0000 7fcda9600640 1 -- 172.21.15.33:0/2719318760 <== mgr.4100 v2:172.21.15.33:6832/199212674 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252551 (secure 0 0 0) 0x7fcdac06b470 con 0x7fcd8c06bc00 2024-03-28T01:08:58.890 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:08:58.892 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-03-28T01:08:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.888+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd8c06bc00 msgr2=0x7fcd8c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.888+0000 7fcdb2800640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd8c06bc00 0x7fcd8c06e0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcda0005880 tx=0x7fcda0009040 comp rx=0 tx=0).stop 2024-03-28T01:08:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.888+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 msgr2=0x7fcdac1448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.888+0000 7fcdb2800640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1448c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd980098c0 tx=0x7fcd980046c0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 shutdown_connections 2024-03-28T01:08:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7fcd8c06bc00 0x7fcd8c06e0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdac148b20 0x7fcdac1dc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdac13f500 0x7fcdac1448c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 --2- 172.21.15.33:0/2719318760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcdac0b2130 0x7fcdac1458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:58.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 >> 172.21.15.33:0/2719318760 conn(0x7fcdac0b89c0 msgr2=0x7fcdac13dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:58.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 shutdown_connections 2024-03-28T01:08:58.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:58.889+0000 7fcdb2800640 1 -- 172.21.15.33:0/2719318760 wait complete. 2024-03-28T01:08:58.911 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-03-28T01:08:58.439186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":741183,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":125,"num_read_kb":109,"num_write":358,"num_write_kb":2667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1346,"ondisk_log_size":1346,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219592,"kb_used_data":3464,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755128,"statfs":{"total":773094113280,"available":772869251072,"internally_reserved":0,"allocated":3547136,"data_stored":2198102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005258"},"pg_stats":[{"pgid":"3.3f","version":"23'18","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:53.886181+0000","last_change":"2024-03-28T01:08:53.886244+0000","last_active":"2024-03-28T01:08:53.886181+0000","last_peered":"2024-03-28T01:08:53.886181+0000","last_clean":"2024-03-28T01:08:53.886181+0000","last_became_active":"2024-03-28T01:06:22.747255+0000","last_became_peered":"2024-03-28T01:06:22.747255+0000","last_unstale":"2024-03-28T01:08:53.886181+0000","last_undegraded":"2024-03-28T01:08:53.886181+0000","last_fullsized":"2024-03-28T01:08:53.886181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'18","last_scrub_stamp":"2024-03-28T01:08:53.885995+0000","last_deep_scrub":"23'18","last_deep_scrub_stamp":"2024-03-28T01:08:53.885995+0000","last_clean_scrub_stamp":"2024-03-28T01:08:53.885995+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:56:03.563076+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:44.023955+0000","last_change":"2024-03-28T01:08:44.024022+0000","last_active":"2024-03-28T01:08:44.023955+0000","last_peered":"2024-03-28T01:08:44.023955+0000","last_clean":"2024-03-28T01:08:44.023955+0000","last_became_active":"2024-03-28T01:06:22.732142+0000","last_became_peered":"2024-03-28T01:06:22.732142+0000","last_unstale":"2024-03-28T01:08:44.023955+0000","last_undegraded":"2024-03-28T01:08:44.023955+0000","last_fullsized":"2024-03-28T01:08:44.023955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-03-28T01:08:44.023729+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-03-28T01:08:44.023729+0000","last_clean_scrub_stamp":"2024-03-28T01:08:44.023729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:28:45.308889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'15","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:36.138592+0000","last_change":"2024-03-28T01:08:36.138637+0000","last_active":"2024-03-28T01:08:36.138592+0000","last_peered":"2024-03-28T01:08:36.138592+0000","last_clean":"2024-03-28T01:08:36.138592+0000","last_became_active":"2024-03-28T01:06:22.718426+0000","last_became_peered":"2024-03-28T01:06:22.718426+0000","last_unstale":"2024-03-28T01:08:36.138592+0000","last_undegraded":"2024-03-28T01:08:36.138592+0000","last_fullsized":"2024-03-28T01:08:36.138592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'15","last_scrub_stamp":"2024-03-28T01:08:36.138494+0000","last_deep_scrub":"23'15","last_deep_scrub_stamp":"2024-03-28T01:08:36.138494+0000","last_clean_scrub_stamp":"2024-03-28T01:08:36.138494+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:15:34.419209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:49.378904+0000","last_change":"2024-03-28T01:08:49.378969+0000","last_active":"2024-03-28T01:08:49.378904+0000","last_peered":"2024-03-28T01:08:49.378904+0000","last_clean":"2024-03-28T01:08:49.378904+0000","last_became_active":"2024-03-28T01:06:22.750664+0000","last_became_peered":"2024-03-28T01:06:22.750664+0000","last_unstale":"2024-03-28T01:08:49.378904+0000","last_undegraded":"2024-03-28T01:08:49.378904+0000","last_fullsized":"2024-03-28T01:08:49.378904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:49.378713+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:49.378713+0000","last_clean_scrub_stamp":"2024-03-28T01:08:49.378713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:08:59.597150+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:50.873280+0000","last_change":"2024-03-28T01:08:50.873280+0000","last_active":"2024-03-28T01:08:50.873280+0000","last_peered":"2024-03-28T01:08:50.873280+0000","last_clean":"2024-03-28T01:08:50.873280+0000","last_became_active":"2024-03-28T01:06:22.745718+0000","last_became_peered":"2024-03-28T01:06:22.745718+0000","last_unstale":"2024-03-28T01:08:50.873280+0000","last_undegraded":"2024-03-28T01:08:50.873280+0000","last_fullsized":"2024-03-28T01:08:50.873280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:50.873028+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:50.873028+0000","last_clean_scrub_stamp":"2024-03-28T01:08:50.873028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:56:58.185098+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'12","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:43.369225+0000","last_change":"2024-03-28T01:08:43.369225+0000","last_active":"2024-03-28T01:08:43.369225+0000","last_peered":"2024-03-28T01:08:43.369225+0000","last_clean":"2024-03-28T01:08:43.369225+0000","last_became_active":"2024-03-28T01:06:22.736211+0000","last_became_peered":"2024-03-28T01:06:22.736211+0000","last_unstale":"2024-03-28T01:08:43.369225+0000","last_undegraded":"2024-03-28T01:08:43.369225+0000","last_fullsized":"2024-03-28T01:08:43.369225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:43.368869+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:43.368869+0000","last_clean_scrub_stamp":"2024-03-28T01:08:43.368869+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:13:08.634964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:48.835130+0000","last_change":"2024-03-28T01:08:48.835130+0000","last_active":"2024-03-28T01:08:48.835130+0000","last_peered":"2024-03-28T01:08:48.835130+0000","last_clean":"2024-03-28T01:08:48.835130+0000","last_became_active":"2024-03-28T01:06:22.744576+0000","last_became_peered":"2024-03-28T01:06:22.744576+0000","last_unstale":"2024-03-28T01:08:48.835130+0000","last_undegraded":"2024-03-28T01:08:48.835130+0000","last_fullsized":"2024-03-28T01:08:48.835130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:48.834856+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:48.834856+0000","last_clean_scrub_stamp":"2024-03-28T01:08:48.834856+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:37:16.792694+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'18","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:45.076089+0000","last_change":"2024-03-28T01:08:45.076089+0000","last_active":"2024-03-28T01:08:45.076089+0000","last_peered":"2024-03-28T01:08:45.076089+0000","last_clean":"2024-03-28T01:08:45.076089+0000","last_became_active":"2024-03-28T01:06:22.739985+0000","last_became_peered":"2024-03-28T01:06:22.739985+0000","last_unstale":"2024-03-28T01:08:45.076089+0000","last_undegraded":"2024-03-28T01:08:45.076089+0000","last_fullsized":"2024-03-28T01:08:45.076089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'18","last_scrub_stamp":"2024-03-28T01:08:45.075848+0000","last_deep_scrub":"23'18","last_deep_scrub_stamp":"2024-03-28T01:08:45.075848+0000","last_clean_scrub_stamp":"2024-03-28T01:08:45.075848+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:13:10.341942+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'22","reported_seq":50,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:52.865513+0000","last_change":"2024-03-28T01:08:52.865582+0000","last_active":"2024-03-28T01:08:52.865513+0000","last_peered":"2024-03-28T01:08:52.865513+0000","last_clean":"2024-03-28T01:08:52.865513+0000","last_became_active":"2024-03-28T01:06:22.748056+0000","last_became_peered":"2024-03-28T01:06:22.748056+0000","last_unstale":"2024-03-28T01:08:52.865513+0000","last_undegraded":"2024-03-28T01:08:52.865513+0000","last_fullsized":"2024-03-28T01:08:52.865513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'22","last_scrub_stamp":"2024-03-28T01:08:52.865285+0000","last_deep_scrub":"23'22","last_deep_scrub_stamp":"2024-03-28T01:08:52.865285+0000","last_clean_scrub_stamp":"2024-03-28T01:08:52.865285+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:06:43.214486+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'34","reported_seq":62,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:47.795284+0000","last_change":"2024-03-28T01:08:47.795284+0000","last_active":"2024-03-28T01:08:47.795284+0000","last_peered":"2024-03-28T01:08:47.795284+0000","last_clean":"2024-03-28T01:08:47.795284+0000","last_became_active":"2024-03-28T01:06:22.734231+0000","last_became_peered":"2024-03-28T01:06:22.734231+0000","last_unstale":"2024-03-28T01:08:47.795284+0000","last_undegraded":"2024-03-28T01:08:47.795284+0000","last_fullsized":"2024-03-28T01:08:47.795284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'34","last_scrub_stamp":"2024-03-28T01:08:47.795027+0000","last_deep_scrub":"23'34","last_deep_scrub_stamp":"2024-03-28T01:08:47.795027+0000","last_clean_scrub_stamp":"2024-03-28T01:08:47.795027+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:42:14.693139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'12","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:48.751402+0000","last_change":"2024-03-28T01:08:48.751402+0000","last_active":"2024-03-28T01:08:48.751402+0000","last_peered":"2024-03-28T01:08:48.751402+0000","last_clean":"2024-03-28T01:08:48.751402+0000","last_became_active":"2024-03-28T01:06:22.739456+0000","last_became_peered":"2024-03-28T01:06:22.739456+0000","last_unstale":"2024-03-28T01:08:48.751402+0000","last_undegraded":"2024-03-28T01:08:48.751402+0000","last_fullsized":"2024-03-28T01:08:48.751402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:48.751123+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:48.751123+0000","last_clean_scrub_stamp":"2024-03-28T01:08:48.751123+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:28:50.036283+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'12","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.707966+0000","last_change":"2024-03-28T01:08:40.707966+0000","last_active":"2024-03-28T01:08:40.707966+0000","last_peered":"2024-03-28T01:08:40.707966+0000","last_clean":"2024-03-28T01:08:40.707966+0000","last_became_active":"2024-03-28T01:06:22.739116+0000","last_became_peered":"2024-03-28T01:06:22.739116+0000","last_unstale":"2024-03-28T01:08:40.707966+0000","last_undegraded":"2024-03-28T01:08:40.707966+0000","last_fullsized":"2024-03-28T01:08:40.707966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:40.707701+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:40.707701+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.707701+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:49:44.906267+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'20","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:47.133068+0000","last_change":"2024-03-28T01:08:47.133068+0000","last_active":"2024-03-28T01:08:47.133068+0000","last_peered":"2024-03-28T01:08:47.133068+0000","last_clean":"2024-03-28T01:08:47.133068+0000","last_became_active":"2024-03-28T01:06:22.740245+0000","last_became_peered":"2024-03-28T01:06:22.740245+0000","last_unstale":"2024-03-28T01:08:47.133068+0000","last_undegraded":"2024-03-28T01:08:47.133068+0000","last_fullsized":"2024-03-28T01:08:47.133068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'20","last_scrub_stamp":"2024-03-28T01:08:47.132814+0000","last_deep_scrub":"23'20","last_deep_scrub_stamp":"2024-03-28T01:08:47.132814+0000","last_clean_scrub_stamp":"2024-03-28T01:08:47.132814+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:30:56.015238+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:37.189474+0000","last_change":"2024-03-28T01:08:37.189474+0000","last_active":"2024-03-28T01:08:37.189474+0000","last_peered":"2024-03-28T01:08:37.189474+0000","last_clean":"2024-03-28T01:08:37.189474+0000","last_became_active":"2024-03-28T01:06:22.721135+0000","last_became_peered":"2024-03-28T01:06:22.721135+0000","last_unstale":"2024-03-28T01:08:37.189474+0000","last_undegraded":"2024-03-28T01:08:37.189474+0000","last_fullsized":"2024-03-28T01:08:37.189474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:37.189346+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:37.189346+0000","last_clean_scrub_stamp":"2024-03-28T01:08:37.189346+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:49:20.850318+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'14","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.994867+0000","last_change":"2024-03-28T01:08:42.994867+0000","last_active":"2024-03-28T01:08:42.994867+0000","last_peered":"2024-03-28T01:08:42.994867+0000","last_clean":"2024-03-28T01:08:42.994867+0000","last_became_active":"2024-03-28T01:06:22.728924+0000","last_became_peered":"2024-03-28T01:06:22.728924+0000","last_unstale":"2024-03-28T01:08:42.994867+0000","last_undegraded":"2024-03-28T01:08:42.994867+0000","last_fullsized":"2024-03-28T01:08:42.994867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:42.994453+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:42.994453+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.994453+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:29:40.256162+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'21","reported_seq":49,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.402851+0000","last_change":"2024-03-28T01:08:42.402851+0000","last_active":"2024-03-28T01:08:42.402851+0000","last_peered":"2024-03-28T01:08:42.402851+0000","last_clean":"2024-03-28T01:08:42.402851+0000","last_became_active":"2024-03-28T01:06:22.736207+0000","last_became_peered":"2024-03-28T01:06:22.736207+0000","last_unstale":"2024-03-28T01:08:42.402851+0000","last_undegraded":"2024-03-28T01:08:42.402851+0000","last_fullsized":"2024-03-28T01:08:42.402851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'21","last_scrub_stamp":"2024-03-28T01:08:42.402594+0000","last_deep_scrub":"23'21","last_deep_scrub_stamp":"2024-03-28T01:08:42.402594+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.402594+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:01:38.545385+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:48.416827+0000","last_change":"2024-03-28T01:08:48.416886+0000","last_active":"2024-03-28T01:08:48.416827+0000","last_peered":"2024-03-28T01:08:48.416827+0000","last_clean":"2024-03-28T01:08:48.416827+0000","last_became_active":"2024-03-28T01:06:22.747787+0000","last_became_peered":"2024-03-28T01:06:22.747787+0000","last_unstale":"2024-03-28T01:08:48.416827+0000","last_undegraded":"2024-03-28T01:08:48.416827+0000","last_fullsized":"2024-03-28T01:08:48.416827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:48.416619+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:48.416619+0000","last_clean_scrub_stamp":"2024-03-28T01:08:48.416619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:51:47.229832+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'20","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:38.443449+0000","last_change":"2024-03-28T01:08:38.443449+0000","last_active":"2024-03-28T01:08:38.443449+0000","last_peered":"2024-03-28T01:08:38.443449+0000","last_clean":"2024-03-28T01:08:38.443449+0000","last_became_active":"2024-03-28T01:06:22.723418+0000","last_became_peered":"2024-03-28T01:06:22.723418+0000","last_unstale":"2024-03-28T01:08:38.443449+0000","last_undegraded":"2024-03-28T01:08:38.443449+0000","last_fullsized":"2024-03-28T01:08:38.443449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'20","last_scrub_stamp":"2024-03-28T01:08:38.443197+0000","last_deep_scrub":"23'20","last_deep_scrub_stamp":"2024-03-28T01:08:38.443197+0000","last_clean_scrub_stamp":"2024-03-28T01:08:38.443197+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:53:23.006194+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'14","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:46.039191+0000","last_change":"2024-03-28T01:08:46.039255+0000","last_active":"2024-03-28T01:08:46.039191+0000","last_peered":"2024-03-28T01:08:46.039191+0000","last_clean":"2024-03-28T01:08:46.039191+0000","last_became_active":"2024-03-28T01:07:03.937565+0000","last_became_peered":"2024-03-28T01:07:03.937565+0000","last_unstale":"2024-03-28T01:08:46.039191+0000","last_undegraded":"2024-03-28T01:08:46.039191+0000","last_fullsized":"2024-03-28T01:08:46.039191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:46.038989+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:46.038989+0000","last_clean_scrub_stamp":"2024-03-28T01:08:46.038989+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:13:11.305084+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:45.719445+0000","last_change":"2024-03-28T01:08:45.719509+0000","last_active":"2024-03-28T01:08:45.719445+0000","last_peered":"2024-03-28T01:08:45.719445+0000","last_clean":"2024-03-28T01:08:45.719445+0000","last_became_active":"2024-03-28T01:06:22.731616+0000","last_became_peered":"2024-03-28T01:06:22.731616+0000","last_unstale":"2024-03-28T01:08:45.719445+0000","last_undegraded":"2024-03-28T01:08:45.719445+0000","last_fullsized":"2024-03-28T01:08:45.719445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:45.719230+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:45.719230+0000","last_clean_scrub_stamp":"2024-03-28T01:08:45.719230+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:36:57.206954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'24","reported_seq":52,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.197623+0000","last_change":"2024-03-28T01:08:40.197623+0000","last_active":"2024-03-28T01:08:40.197623+0000","last_peered":"2024-03-28T01:08:40.197623+0000","last_clean":"2024-03-28T01:08:40.197623+0000","last_became_active":"2024-03-28T01:06:22.734428+0000","last_became_peered":"2024-03-28T01:06:22.734428+0000","last_unstale":"2024-03-28T01:08:40.197623+0000","last_undegraded":"2024-03-28T01:08:40.197623+0000","last_fullsized":"2024-03-28T01:08:40.197623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'24","last_scrub_stamp":"2024-03-28T01:08:40.197221+0000","last_deep_scrub":"23'24","last_deep_scrub_stamp":"2024-03-28T01:08:40.197221+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.197221+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:36:51.684945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'20","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.412027+0000","last_change":"2024-03-28T01:08:42.412091+0000","last_active":"2024-03-28T01:08:42.412027+0000","last_peered":"2024-03-28T01:08:42.412027+0000","last_clean":"2024-03-28T01:08:42.412027+0000","last_became_active":"2024-03-28T01:07:03.972014+0000","last_became_peered":"2024-03-28T01:07:03.972014+0000","last_unstale":"2024-03-28T01:08:42.412027+0000","last_undegraded":"2024-03-28T01:08:42.412027+0000","last_fullsized":"2024-03-28T01:08:42.412027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'20","last_scrub_stamp":"2024-03-28T01:08:42.411823+0000","last_deep_scrub":"23'20","last_deep_scrub_stamp":"2024-03-28T01:08:42.411823+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.411823+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:25:53.817047+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:41.051904+0000","last_change":"2024-03-28T01:08:41.051971+0000","last_active":"2024-03-28T01:08:41.051904+0000","last_peered":"2024-03-28T01:08:41.051904+0000","last_clean":"2024-03-28T01:08:41.051904+0000","last_became_active":"2024-03-28T01:06:22.734151+0000","last_became_peered":"2024-03-28T01:06:22.734151+0000","last_unstale":"2024-03-28T01:08:41.051904+0000","last_undegraded":"2024-03-28T01:08:41.051904+0000","last_fullsized":"2024-03-28T01:08:41.051904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:41.051718+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:41.051718+0000","last_clean_scrub_stamp":"2024-03-28T01:08:41.051718+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:20:26.308937+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:46.382881+0000","last_change":"2024-03-28T01:08:46.382998+0000","last_active":"2024-03-28T01:08:46.382881+0000","last_peered":"2024-03-28T01:08:46.382881+0000","last_clean":"2024-03-28T01:08:46.382881+0000","last_became_active":"2024-03-28T01:06:22.747251+0000","last_became_peered":"2024-03-28T01:06:22.747251+0000","last_unstale":"2024-03-28T01:08:46.382881+0000","last_undegraded":"2024-03-28T01:08:46.382881+0000","last_fullsized":"2024-03-28T01:08:46.382881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:46.382640+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:46.382640+0000","last_clean_scrub_stamp":"2024-03-28T01:08:46.382640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:21:21.801967+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'4","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:43.042415+0000","last_change":"2024-03-28T01:08:43.042474+0000","last_active":"2024-03-28T01:08:43.042415+0000","last_peered":"2024-03-28T01:08:43.042415+0000","last_clean":"2024-03-28T01:08:43.042415+0000","last_became_active":"2024-03-28T01:06:22.739235+0000","last_became_peered":"2024-03-28T01:06:22.739235+0000","last_unstale":"2024-03-28T01:08:43.042415+0000","last_undegraded":"2024-03-28T01:08:43.042415+0000","last_fullsized":"2024-03-28T01:08:43.042415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-03-28T01:08:43.042226+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-03-28T01:08:43.042226+0000","last_clean_scrub_stamp":"2024-03-28T01:08:43.042226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:44:15.565702+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'32","reported_seq":60,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:44.691740+0000","last_change":"2024-03-28T01:08:44.691931+0000","last_active":"2024-03-28T01:08:44.691740+0000","last_peered":"2024-03-28T01:08:44.691740+0000","last_clean":"2024-03-28T01:08:44.691740+0000","last_became_active":"2024-03-28T01:06:22.731069+0000","last_became_peered":"2024-03-28T01:06:22.731069+0000","last_unstale":"2024-03-28T01:08:44.691740+0000","last_undegraded":"2024-03-28T01:08:44.691740+0000","last_fullsized":"2024-03-28T01:08:44.691740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'32","last_scrub_stamp":"2024-03-28T01:08:44.691514+0000","last_deep_scrub":"23'32","last_deep_scrub_stamp":"2024-03-28T01:08:44.691514+0000","last_clean_scrub_stamp":"2024-03-28T01:08:44.691514+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:41:16.071672+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'12","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:45.010954+0000","last_change":"2024-03-28T01:08:45.011018+0000","last_active":"2024-03-28T01:08:45.010954+0000","last_peered":"2024-03-28T01:08:45.010954+0000","last_clean":"2024-03-28T01:08:45.010954+0000","last_became_active":"2024-03-28T01:06:22.739858+0000","last_became_peered":"2024-03-28T01:06:22.739858+0000","last_unstale":"2024-03-28T01:08:45.010954+0000","last_undegraded":"2024-03-28T01:08:45.010954+0000","last_fullsized":"2024-03-28T01:08:45.010954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:45.010755+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:45.010755+0000","last_clean_scrub_stamp":"2024-03-28T01:08:45.010755+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:01:41.153545+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'8","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.052464+0000","last_change":"2024-03-28T01:08:40.052522+0000","last_active":"2024-03-28T01:08:40.052464+0000","last_peered":"2024-03-28T01:08:40.052464+0000","last_clean":"2024-03-28T01:08:40.052464+0000","last_became_active":"2024-03-28T01:06:22.733689+0000","last_became_peered":"2024-03-28T01:06:22.733689+0000","last_unstale":"2024-03-28T01:08:40.052464+0000","last_undegraded":"2024-03-28T01:08:40.052464+0000","last_fullsized":"2024-03-28T01:08:40.052464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-03-28T01:08:40.052278+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-03-28T01:08:40.052278+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.052278+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:49:20.665226+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'14","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.220207+0000","last_change":"2024-03-28T01:08:42.220349+0000","last_active":"2024-03-28T01:08:42.220207+0000","last_peered":"2024-03-28T01:08:42.220207+0000","last_clean":"2024-03-28T01:08:42.220207+0000","last_became_active":"2024-03-28T01:07:03.941729+0000","last_became_peered":"2024-03-28T01:07:03.941729+0000","last_unstale":"2024-03-28T01:08:42.220207+0000","last_undegraded":"2024-03-28T01:08:42.220207+0000","last_fullsized":"2024-03-28T01:08:42.220207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:42.219975+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:42.219975+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.219975+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:28:43.505135+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'16","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.468572+0000","last_change":"2024-03-28T01:08:40.468636+0000","last_active":"2024-03-28T01:08:40.468572+0000","last_peered":"2024-03-28T01:08:40.468572+0000","last_clean":"2024-03-28T01:08:40.468572+0000","last_became_active":"2024-03-28T01:07:03.929334+0000","last_became_peered":"2024-03-28T01:07:03.929334+0000","last_unstale":"2024-03-28T01:08:40.468572+0000","last_undegraded":"2024-03-28T01:08:40.468572+0000","last_fullsized":"2024-03-28T01:08:40.468572+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:40.468355+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:40.468355+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.468355+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:15:38.749069+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"23'24","reported_seq":52,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.492482+0000","last_change":"2024-03-28T01:08:39.492541+0000","last_active":"2024-03-28T01:08:39.492482+0000","last_peered":"2024-03-28T01:08:39.492482+0000","last_clean":"2024-03-28T01:08:39.492482+0000","last_became_active":"2024-03-28T01:06:22.733111+0000","last_became_peered":"2024-03-28T01:06:22.733111+0000","last_unstale":"2024-03-28T01:08:39.492482+0000","last_undegraded":"2024-03-28T01:08:39.492482+0000","last_fullsized":"2024-03-28T01:08:39.492482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'24","last_scrub_stamp":"2024-03-28T01:08:39.492280+0000","last_deep_scrub":"23'24","last_deep_scrub_stamp":"2024-03-28T01:08:39.492280+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.492280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:21:20.565818+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'18","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:43.684804+0000","last_change":"2024-03-28T01:08:43.684931+0000","last_active":"2024-03-28T01:08:43.684804+0000","last_peered":"2024-03-28T01:08:43.684804+0000","last_clean":"2024-03-28T01:08:43.684804+0000","last_became_active":"2024-03-28T01:06:22.730473+0000","last_became_peered":"2024-03-28T01:06:22.730473+0000","last_unstale":"2024-03-28T01:08:43.684804+0000","last_undegraded":"2024-03-28T01:08:43.684804+0000","last_fullsized":"2024-03-28T01:08:43.684804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'18","last_scrub_stamp":"2024-03-28T01:08:43.684472+0000","last_deep_scrub":"23'18","last_deep_scrub_stamp":"2024-03-28T01:08:43.684472+0000","last_clean_scrub_stamp":"2024-03-28T01:08:43.684472+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:25:55.089696+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'14","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:38.662888+0000","last_change":"2024-03-28T01:08:38.662888+0000","last_active":"2024-03-28T01:08:38.662888+0000","last_peered":"2024-03-28T01:08:38.662888+0000","last_clean":"2024-03-28T01:08:38.662888+0000","last_became_active":"2024-03-28T01:06:22.723509+0000","last_became_peered":"2024-03-28T01:06:22.723509+0000","last_unstale":"2024-03-28T01:08:38.662888+0000","last_undegraded":"2024-03-28T01:08:38.662888+0000","last_fullsized":"2024-03-28T01:08:38.662888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:38.662634+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:38.662634+0000","last_clean_scrub_stamp":"2024-03-28T01:08:38.662634+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:15:10.870186+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:36.960705+0000","last_change":"2024-03-28T01:08:36.960705+0000","last_active":"2024-03-28T01:08:36.960705+0000","last_peered":"2024-03-28T01:08:36.960705+0000","last_clean":"2024-03-28T01:08:36.960705+0000","last_became_active":"2024-03-28T01:06:21.728029+0000","last_became_peered":"2024-03-28T01:06:21.728029+0000","last_unstale":"2024-03-28T01:08:36.960705+0000","last_undegraded":"2024-03-28T01:08:36.960705+0000","last_fullsized":"2024-03-28T01:08:36.960705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2024-03-28T01:08:36.960529+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2024-03-28T01:08:36.960529+0000","last_clean_scrub_stamp":"2024-03-28T01:08:36.960529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:39:54.621094+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.184493+0000","last_change":"2024-03-28T01:08:35.184493+0000","last_active":"2024-03-28T01:08:35.184493+0000","last_peered":"2024-03-28T01:08:35.184493+0000","last_clean":"2024-03-28T01:08:35.184493+0000","last_became_active":"2024-03-28T01:06:22.717697+0000","last_became_peered":"2024-03-28T01:06:22.717697+0000","last_unstale":"2024-03-28T01:08:35.184493+0000","last_undegraded":"2024-03-28T01:08:35.184493+0000","last_fullsized":"2024-03-28T01:08:35.184493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:35.184290+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:35.184290+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.184290+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:45:27.934451+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.707117+0000","last_change":"2024-03-28T01:08:40.707117+0000","last_active":"2024-03-28T01:08:40.707117+0000","last_peered":"2024-03-28T01:08:40.707117+0000","last_clean":"2024-03-28T01:08:40.707117+0000","last_became_active":"2024-03-28T01:06:21.733302+0000","last_became_peered":"2024-03-28T01:06:21.733302+0000","last_unstale":"2024-03-28T01:08:40.707117+0000","last_undegraded":"2024-03-28T01:08:40.707117+0000","last_fullsized":"2024-03-28T01:08:40.707117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2024-03-28T01:08:40.706699+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2024-03-28T01:08:40.706699+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.706699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:58:20.439633+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.891029+0000","last_change":"2024-03-28T01:08:34.891029+0000","last_active":"2024-03-28T01:08:34.891029+0000","last_peered":"2024-03-28T01:08:34.891029+0000","last_clean":"2024-03-28T01:08:34.891029+0000","last_became_active":"2024-03-28T01:06:21.720985+0000","last_became_peered":"2024-03-28T01:06:21.720985+0000","last_unstale":"2024-03-28T01:08:34.891029+0000","last_undegraded":"2024-03-28T01:08:34.891029+0000","last_fullsized":"2024-03-28T01:08:34.891029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:34.890891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:34.890891+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.890891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:53:19.453887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.005961+0000","last_change":"2024-03-28T01:08:39.005961+0000","last_active":"2024-03-28T01:08:39.005961+0000","last_peered":"2024-03-28T01:08:39.005961+0000","last_clean":"2024-03-28T01:08:39.005961+0000","last_became_active":"2024-03-28T01:06:22.729586+0000","last_became_peered":"2024-03-28T01:06:22.729586+0000","last_unstale":"2024-03-28T01:08:39.005961+0000","last_undegraded":"2024-03-28T01:08:39.005961+0000","last_fullsized":"2024-03-28T01:08:39.005961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:39.005713+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:39.005713+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.005713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:41:10.385871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.418202+0000","last_change":"2024-03-28T01:08:40.418202+0000","last_active":"2024-03-28T01:08:40.418202+0000","last_peered":"2024-03-28T01:08:40.418202+0000","last_clean":"2024-03-28T01:08:40.418202+0000","last_became_active":"2024-03-28T01:06:21.732564+0000","last_became_peered":"2024-03-28T01:06:21.732564+0000","last_unstale":"2024-03-28T01:08:40.418202+0000","last_undegraded":"2024-03-28T01:08:40.418202+0000","last_fullsized":"2024-03-28T01:08:40.418202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:40.417942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:40.417942+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.417942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:29:37.679651+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'24","reported_seq":52,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.008223+0000","last_change":"2024-03-28T01:08:42.008223+0000","last_active":"2024-03-28T01:08:42.008223+0000","last_peered":"2024-03-28T01:08:42.008223+0000","last_clean":"2024-03-28T01:08:42.008223+0000","last_became_active":"2024-03-28T01:06:22.723243+0000","last_became_peered":"2024-03-28T01:06:22.723243+0000","last_unstale":"2024-03-28T01:08:42.008223+0000","last_undegraded":"2024-03-28T01:08:42.008223+0000","last_fullsized":"2024-03-28T01:08:42.008223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'24","last_scrub_stamp":"2024-03-28T01:08:42.007949+0000","last_deep_scrub":"23'24","last_deep_scrub_stamp":"2024-03-28T01:08:42.007949+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.007949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:36:53.495674+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.697713+0000","last_change":"2024-03-28T01:08:39.697713+0000","last_active":"2024-03-28T01:08:39.697713+0000","last_peered":"2024-03-28T01:08:39.697713+0000","last_clean":"2024-03-28T01:08:39.697713+0000","last_became_active":"2024-03-28T01:06:21.731939+0000","last_became_peered":"2024-03-28T01:06:21.731939+0000","last_unstale":"2024-03-28T01:08:39.697713+0000","last_undegraded":"2024-03-28T01:08:39.697713+0000","last_fullsized":"2024-03-28T01:08:39.697713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-03-28T01:08:39.697459+0000","last_deep_scrub":"22'1","last_deep_scrub_stamp":"2024-03-28T01:08:39.697459+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.697459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:59:06.234361+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'20","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:47.427269+0000","last_change":"2024-03-28T01:08:47.427269+0000","last_active":"2024-03-28T01:08:47.427269+0000","last_peered":"2024-03-28T01:08:47.427269+0000","last_clean":"2024-03-28T01:08:47.427269+0000","last_became_active":"2024-03-28T01:06:22.747721+0000","last_became_peered":"2024-03-28T01:06:22.747721+0000","last_unstale":"2024-03-28T01:08:47.427269+0000","last_undegraded":"2024-03-28T01:08:47.427269+0000","last_fullsized":"2024-03-28T01:08:47.427269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'20","last_scrub_stamp":"2024-03-28T01:08:47.427008+0000","last_deep_scrub":"23'20","last_deep_scrub_stamp":"2024-03-28T01:08:47.427008+0000","last_clean_scrub_stamp":"2024-03-28T01:08:47.427008+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:11:02.579269+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.442263+0000","last_change":"2024-03-28T01:08:39.442263+0000","last_active":"2024-03-28T01:08:39.442263+0000","last_peered":"2024-03-28T01:08:39.442263+0000","last_clean":"2024-03-28T01:08:39.442263+0000","last_became_active":"2024-03-28T01:06:21.729808+0000","last_became_peered":"2024-03-28T01:06:21.729808+0000","last_unstale":"2024-03-28T01:08:39.442263+0000","last_undegraded":"2024-03-28T01:08:39.442263+0000","last_fullsized":"2024-03-28T01:08:39.442263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:39.442004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:39.442004+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.442004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:36:50.929728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'16","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:41.426654+0000","last_change":"2024-03-28T01:08:41.426741+0000","last_active":"2024-03-28T01:08:41.426654+0000","last_peered":"2024-03-28T01:08:41.426654+0000","last_clean":"2024-03-28T01:08:41.426654+0000","last_became_active":"2024-03-28T01:07:03.970312+0000","last_became_peered":"2024-03-28T01:07:03.970312+0000","last_unstale":"2024-03-28T01:08:41.426654+0000","last_undegraded":"2024-03-28T01:08:41.426654+0000","last_fullsized":"2024-03-28T01:08:41.426654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:41.426459+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:41.426459+0000","last_clean_scrub_stamp":"2024-03-28T01:08:41.426459+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:02:41.235668+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"23'11","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:37.028148+0000","last_change":"2024-03-28T01:08:37.028148+0000","last_active":"2024-03-28T01:08:37.028148+0000","last_peered":"2024-03-28T01:08:37.028148+0000","last_clean":"2024-03-28T01:08:37.028148+0000","last_became_active":"2024-03-28T01:06:21.724346+0000","last_became_peered":"2024-03-28T01:06:21.724346+0000","last_unstale":"2024-03-28T01:08:37.028148+0000","last_undegraded":"2024-03-28T01:08:37.028148+0000","last_fullsized":"2024-03-28T01:08:37.028148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-03-28T01:08:37.027908+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-03-28T01:08:37.027908+0000","last_clean_scrub_stamp":"2024-03-28T01:08:37.027908+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:58:16.760842+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'22","reported_seq":50,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.663055+0000","last_change":"2024-03-28T01:08:39.663055+0000","last_active":"2024-03-28T01:08:39.663055+0000","last_peered":"2024-03-28T01:08:39.663055+0000","last_clean":"2024-03-28T01:08:39.663055+0000","last_became_active":"2024-03-28T01:06:22.738011+0000","last_became_peered":"2024-03-28T01:06:22.738011+0000","last_unstale":"2024-03-28T01:08:39.663055+0000","last_undegraded":"2024-03-28T01:08:39.663055+0000","last_fullsized":"2024-03-28T01:08:39.663055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'22","last_scrub_stamp":"2024-03-28T01:08:39.662794+0000","last_deep_scrub":"23'22","last_deep_scrub_stamp":"2024-03-28T01:08:39.662794+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.662794+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:12:19.263418+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'123","reported_seq":153,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:37.663892+0000","last_change":"2024-03-28T01:08:37.663892+0000","last_active":"2024-03-28T01:08:37.663892+0000","last_peered":"2024-03-28T01:08:37.663892+0000","last_clean":"2024-03-28T01:08:37.663892+0000","last_became_active":"2024-03-28T01:06:21.722658+0000","last_became_peered":"2024-03-28T01:06:21.722658+0000","last_unstale":"2024-03-28T01:08:37.663892+0000","last_undegraded":"2024-03-28T01:08:37.663892+0000","last_fullsized":"2024-03-28T01:08:37.663892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'123","last_scrub_stamp":"2024-03-28T01:08:37.663523+0000","last_deep_scrub":"23'123","last_deep_scrub_stamp":"2024-03-28T01:08:37.663523+0000","last_clean_scrub_stamp":"2024-03-28T01:08:37.663523+0000","objects_scrubbed":4,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:18:09.334261+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148885,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'13","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:47.864180+0000","last_change":"2024-03-28T01:08:47.864180+0000","last_active":"2024-03-28T01:08:47.864180+0000","last_peered":"2024-03-28T01:08:47.864180+0000","last_clean":"2024-03-28T01:08:47.864180+0000","last_became_active":"2024-03-28T01:06:22.742047+0000","last_became_peered":"2024-03-28T01:06:22.742047+0000","last_unstale":"2024-03-28T01:08:47.864180+0000","last_undegraded":"2024-03-28T01:08:47.864180+0000","last_fullsized":"2024-03-28T01:08:47.864180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13","last_scrub_stamp":"2024-03-28T01:08:47.863713+0000","last_deep_scrub":"23'13","last_deep_scrub_stamp":"2024-03-28T01:08:47.863713+0000","last_clean_scrub_stamp":"2024-03-28T01:08:47.863713+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:22:44.800545+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'5","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.230625+0000","last_change":"2024-03-28T01:08:34.230625+0000","last_active":"2024-03-28T01:08:34.230625+0000","last_peered":"2024-03-28T01:08:34.230625+0000","last_clean":"2024-03-28T01:08:34.230625+0000","last_became_active":"2024-03-28T01:06:21.726898+0000","last_became_peered":"2024-03-28T01:06:21.726898+0000","last_unstale":"2024-03-28T01:08:34.230625+0000","last_undegraded":"2024-03-28T01:08:34.230625+0000","last_fullsized":"2024-03-28T01:08:34.230625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-03-28T01:08:34.230383+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-03-28T01:08:34.230383+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.230383+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:59:00.767286+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'19","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:41.234795+0000","last_change":"2024-03-28T01:08:41.234795+0000","last_active":"2024-03-28T01:08:41.234795+0000","last_peered":"2024-03-28T01:08:41.234795+0000","last_clean":"2024-03-28T01:08:41.234795+0000","last_became_active":"2024-03-28T01:06:22.734113+0000","last_became_peered":"2024-03-28T01:06:22.734113+0000","last_unstale":"2024-03-28T01:08:41.234795+0000","last_undegraded":"2024-03-28T01:08:41.234795+0000","last_fullsized":"2024-03-28T01:08:41.234795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-03-28T01:08:41.234283+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-03-28T01:08:41.234283+0000","last_clean_scrub_stamp":"2024-03-28T01:08:41.234283+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:29:38.495992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:38.410039+0000","last_change":"2024-03-28T01:08:38.410039+0000","last_active":"2024-03-28T01:08:38.410039+0000","last_peered":"2024-03-28T01:08:38.410039+0000","last_clean":"2024-03-28T01:08:38.410039+0000","last_became_active":"2024-03-28T01:06:21.729196+0000","last_became_peered":"2024-03-28T01:06:21.729196+0000","last_unstale":"2024-03-28T01:08:38.410039+0000","last_undegraded":"2024-03-28T01:08:38.410039+0000","last_fullsized":"2024-03-28T01:08:38.410039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:38.409782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:38.409782+0000","last_clean_scrub_stamp":"2024-03-28T01:08:38.409782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:33:58.203338+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'30","reported_seq":58,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:38.849710+0000","last_change":"2024-03-28T01:08:38.849710+0000","last_active":"2024-03-28T01:08:38.849710+0000","last_peered":"2024-03-28T01:08:38.849710+0000","last_clean":"2024-03-28T01:08:38.849710+0000","last_became_active":"2024-03-28T01:06:22.741853+0000","last_became_peered":"2024-03-28T01:06:22.741853+0000","last_unstale":"2024-03-28T01:08:38.849710+0000","last_undegraded":"2024-03-28T01:08:38.849710+0000","last_fullsized":"2024-03-28T01:08:38.849710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'30","last_scrub_stamp":"2024-03-28T01:08:38.849448+0000","last_deep_scrub":"23'30","last_deep_scrub_stamp":"2024-03-28T01:08:38.849448+0000","last_clean_scrub_stamp":"2024-03-28T01:08:38.849448+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:02:38.658657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:38.701358+0000","last_change":"2024-03-28T01:08:38.701358+0000","last_active":"2024-03-28T01:08:38.701358+0000","last_peered":"2024-03-28T01:08:38.701358+0000","last_clean":"2024-03-28T01:08:38.701358+0000","last_became_active":"2024-03-28T01:06:21.727312+0000","last_became_peered":"2024-03-28T01:06:21.727312+0000","last_unstale":"2024-03-28T01:08:38.701358+0000","last_undegraded":"2024-03-28T01:08:38.701358+0000","last_fullsized":"2024-03-28T01:08:38.701358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:38.701106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:38.701106+0000","last_clean_scrub_stamp":"2024-03-28T01:08:38.701106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:53:23.264102+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'14","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.670782+0000","last_change":"2024-03-28T01:08:42.670782+0000","last_active":"2024-03-28T01:08:42.670782+0000","last_peered":"2024-03-28T01:08:42.670782+0000","last_clean":"2024-03-28T01:08:42.670782+0000","last_became_active":"2024-03-28T01:06:22.718970+0000","last_became_peered":"2024-03-28T01:06:22.718970+0000","last_unstale":"2024-03-28T01:08:42.670782+0000","last_undegraded":"2024-03-28T01:08:42.670782+0000","last_fullsized":"2024-03-28T01:08:42.670782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:42.670529+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:42.670529+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.670529+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:02:42.479738+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'12","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.404464+0000","last_change":"2024-03-28T01:08:34.404464+0000","last_active":"2024-03-28T01:08:34.404464+0000","last_peered":"2024-03-28T01:08:34.404464+0000","last_clean":"2024-03-28T01:08:34.404464+0000","last_became_active":"2024-03-28T01:06:21.719130+0000","last_became_peered":"2024-03-28T01:06:21.719130+0000","last_unstale":"2024-03-28T01:08:34.404464+0000","last_undegraded":"2024-03-28T01:08:34.404464+0000","last_fullsized":"2024-03-28T01:08:34.404464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:34.404344+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:34.404344+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.404344+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:15:06.611896+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'12","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:46.832291+0000","last_change":"2024-03-28T01:08:46.832357+0000","last_active":"2024-03-28T01:08:46.832291+0000","last_peered":"2024-03-28T01:08:46.832291+0000","last_clean":"2024-03-28T01:08:46.832291+0000","last_became_active":"2024-03-28T01:06:22.733201+0000","last_became_peered":"2024-03-28T01:06:22.733201+0000","last_unstale":"2024-03-28T01:08:46.832291+0000","last_undegraded":"2024-03-28T01:08:46.832291+0000","last_fullsized":"2024-03-28T01:08:46.832291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:46.832104+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:46.832104+0000","last_clean_scrub_stamp":"2024-03-28T01:08:46.832104+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:42:47.173461+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.646992+0000","last_change":"2024-03-28T01:08:35.646992+0000","last_active":"2024-03-28T01:08:35.646992+0000","last_peered":"2024-03-28T01:08:35.646992+0000","last_clean":"2024-03-28T01:08:35.646992+0000","last_became_active":"2024-03-28T01:06:21.721577+0000","last_became_peered":"2024-03-28T01:06:21.721577+0000","last_unstale":"2024-03-28T01:08:35.646992+0000","last_undegraded":"2024-03-28T01:08:35.646992+0000","last_fullsized":"2024-03-28T01:08:35.646992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-03-28T01:08:35.646741+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-03-28T01:08:35.646741+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.646741+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:04:15.754328+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:49.846414+0000","last_change":"2024-03-28T01:08:49.846522+0000","last_active":"2024-03-28T01:08:49.846414+0000","last_peered":"2024-03-28T01:08:49.846414+0000","last_clean":"2024-03-28T01:08:49.846414+0000","last_became_active":"2024-03-28T01:06:22.745268+0000","last_became_peered":"2024-03-28T01:06:22.745268+0000","last_unstale":"2024-03-28T01:08:49.846414+0000","last_undegraded":"2024-03-28T01:08:49.846414+0000","last_fullsized":"2024-03-28T01:08:49.846414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:49.846186+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:49.846186+0000","last_clean_scrub_stamp":"2024-03-28T01:08:49.846186+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:21:25.265513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'3","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:36.424276+0000","last_change":"2024-03-28T01:08:36.424276+0000","last_active":"2024-03-28T01:08:36.424276+0000","last_peered":"2024-03-28T01:08:36.424276+0000","last_clean":"2024-03-28T01:08:36.424276+0000","last_became_active":"2024-03-28T01:06:21.722190+0000","last_became_peered":"2024-03-28T01:06:21.722190+0000","last_unstale":"2024-03-28T01:08:36.424276+0000","last_undegraded":"2024-03-28T01:08:36.424276+0000","last_fullsized":"2024-03-28T01:08:36.424276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-03-28T01:08:36.424028+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-03-28T01:08:36.424028+0000","last_clean_scrub_stamp":"2024-03-28T01:08:36.424028+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:49:40.622594+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'18","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:54.866340+0000","last_change":"2024-03-28T01:08:54.866401+0000","last_active":"2024-03-28T01:08:54.866340+0000","last_peered":"2024-03-28T01:08:54.866340+0000","last_clean":"2024-03-28T01:08:54.866340+0000","last_became_active":"2024-03-28T01:06:22.750347+0000","last_became_peered":"2024-03-28T01:06:22.750347+0000","last_unstale":"2024-03-28T01:08:54.866340+0000","last_undegraded":"2024-03-28T01:08:54.866340+0000","last_fullsized":"2024-03-28T01:08:54.866340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'18","last_scrub_stamp":"2024-03-28T01:08:54.866142+0000","last_deep_scrub":"23'18","last_deep_scrub_stamp":"2024-03-28T01:08:54.866142+0000","last_clean_scrub_stamp":"2024-03-28T01:08:54.866142+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:34:24.878135+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'2","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.057893+0000","last_change":"2024-03-28T01:08:35.057893+0000","last_active":"2024-03-28T01:08:35.057893+0000","last_peered":"2024-03-28T01:08:35.057893+0000","last_clean":"2024-03-28T01:08:35.057893+0000","last_became_active":"2024-03-28T01:06:21.717832+0000","last_became_peered":"2024-03-28T01:06:21.717832+0000","last_unstale":"2024-03-28T01:08:35.057893+0000","last_undegraded":"2024-03-28T01:08:35.057893+0000","last_fullsized":"2024-03-28T01:08:35.057893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-03-28T01:08:35.057718+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-03-28T01:08:35.057718+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.057718+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:53:19.620715+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'14","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:50.330191+0000","last_change":"2024-03-28T01:08:50.330262+0000","last_active":"2024-03-28T01:08:50.330191+0000","last_peered":"2024-03-28T01:08:50.330191+0000","last_clean":"2024-03-28T01:08:50.330191+0000","last_became_active":"2024-03-28T01:06:22.750133+0000","last_became_peered":"2024-03-28T01:06:22.750133+0000","last_unstale":"2024-03-28T01:08:50.330191+0000","last_undegraded":"2024-03-28T01:08:50.330191+0000","last_fullsized":"2024-03-28T01:08:50.330191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:50.330008+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:50.330008+0000","last_clean_scrub_stamp":"2024-03-28T01:08:50.330008+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:39:12.059387+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:37.400695+0000","last_change":"2024-03-28T01:08:37.400695+0000","last_active":"2024-03-28T01:08:37.400695+0000","last_peered":"2024-03-28T01:08:37.400695+0000","last_clean":"2024-03-28T01:08:37.400695+0000","last_became_active":"2024-03-28T01:06:21.732835+0000","last_became_peered":"2024-03-28T01:06:21.732835+0000","last_unstale":"2024-03-28T01:08:37.400695+0000","last_undegraded":"2024-03-28T01:08:37.400695+0000","last_fullsized":"2024-03-28T01:08:37.400695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:37.400471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:37.400471+0000","last_clean_scrub_stamp":"2024-03-28T01:08:37.400471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:18:09.071209+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'14","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:51.868565+0000","last_change":"2024-03-28T01:08:51.868629+0000","last_active":"2024-03-28T01:08:51.868565+0000","last_peered":"2024-03-28T01:08:51.868565+0000","last_clean":"2024-03-28T01:08:51.868565+0000","last_became_active":"2024-03-28T01:06:22.747976+0000","last_became_peered":"2024-03-28T01:06:22.747976+0000","last_unstale":"2024-03-28T01:08:51.868565+0000","last_undegraded":"2024-03-28T01:08:51.868565+0000","last_fullsized":"2024-03-28T01:08:51.868565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:51.868288+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:51.868288+0000","last_clean_scrub_stamp":"2024-03-28T01:08:51.868288+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:51:50.681501+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.861324+0000","last_change":"2024-03-28T01:08:39.861406+0000","last_active":"2024-03-28T01:08:39.861324+0000","last_peered":"2024-03-28T01:08:39.861324+0000","last_clean":"2024-03-28T01:08:39.861324+0000","last_became_active":"2024-03-28T01:06:11.768737+0000","last_became_peered":"2024-03-28T01:06:11.768737+0000","last_unstale":"2024-03-28T01:08:39.861324+0000","last_undegraded":"2024-03-28T01:08:39.861324+0000","last_fullsized":"2024-03-28T01:08:39.861324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-03-28T01:08:39.861140+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-03-28T01:08:39.861140+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.861140+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:25:51.266364+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:36.419036+0000","last_change":"2024-03-28T01:08:36.419089+0000","last_active":"2024-03-28T01:08:36.419036+0000","last_peered":"2024-03-28T01:08:36.419036+0000","last_clean":"2024-03-28T01:08:36.419036+0000","last_became_active":"2024-03-28T01:06:21.722210+0000","last_became_peered":"2024-03-28T01:06:21.722210+0000","last_unstale":"2024-03-28T01:08:36.419036+0000","last_undegraded":"2024-03-28T01:08:36.419036+0000","last_fullsized":"2024-03-28T01:08:36.419036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:36.418834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:36.418834+0000","last_clean_scrub_stamp":"2024-03-28T01:08:36.418834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:49:20.079806+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.085578+0000","last_change":"2024-03-28T01:08:42.085637+0000","last_active":"2024-03-28T01:08:42.085578+0000","last_peered":"2024-03-28T01:08:42.085578+0000","last_clean":"2024-03-28T01:08:42.085578+0000","last_became_active":"2024-03-28T01:06:22.738757+0000","last_became_peered":"2024-03-28T01:06:22.738757+0000","last_unstale":"2024-03-28T01:08:42.085578+0000","last_undegraded":"2024-03-28T01:08:42.085578+0000","last_fullsized":"2024-03-28T01:08:42.085578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:42.085386+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:42.085386+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.085386+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:29:39.347095+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:45.826592+0000","last_change":"2024-03-28T01:08:45.826645+0000","last_active":"2024-03-28T01:08:45.826592+0000","last_peered":"2024-03-28T01:08:45.826592+0000","last_clean":"2024-03-28T01:08:45.826592+0000","last_became_active":"2024-03-28T01:06:21.736227+0000","last_became_peered":"2024-03-28T01:06:21.736227+0000","last_unstale":"2024-03-28T01:08:45.826592+0000","last_undegraded":"2024-03-28T01:08:45.826592+0000","last_fullsized":"2024-03-28T01:08:45.826592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:45.826366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:45.826366+0000","last_clean_scrub_stamp":"2024-03-28T01:08:45.826366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:01:41.969157+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.205270+0000","last_change":"2024-03-28T01:08:39.205458+0000","last_active":"2024-03-28T01:08:39.205270+0000","last_peered":"2024-03-28T01:08:39.205270+0000","last_clean":"2024-03-28T01:08:39.205270+0000","last_became_active":"2024-03-28T01:06:22.729358+0000","last_became_peered":"2024-03-28T01:06:22.729358+0000","last_unstale":"2024-03-28T01:08:39.205270+0000","last_undegraded":"2024-03-28T01:08:39.205270+0000","last_fullsized":"2024-03-28T01:08:39.205270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:39.205050+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:39.205050+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.205050+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:33:58.998606+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'10","reported_seq":49,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.387134+0000","last_change":"2024-03-28T01:08:35.387179+0000","last_active":"2024-03-28T01:08:35.387134+0000","last_peered":"2024-03-28T01:08:35.387134+0000","last_clean":"2024-03-28T01:08:35.387134+0000","last_became_active":"2024-03-28T01:06:21.720419+0000","last_became_peered":"2024-03-28T01:06:21.720419+0000","last_unstale":"2024-03-28T01:08:35.387134+0000","last_undegraded":"2024-03-28T01:08:35.387134+0000","last_fullsized":"2024-03-28T01:08:35.387134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:35.387008+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:35.387008+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.387008+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:12:14.987632+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'17","reported_seq":45,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.027907+0000","last_change":"2024-03-28T01:08:40.027967+0000","last_active":"2024-03-28T01:08:40.027907+0000","last_peered":"2024-03-28T01:08:40.027907+0000","last_clean":"2024-03-28T01:08:40.027907+0000","last_became_active":"2024-03-28T01:06:22.719869+0000","last_became_peered":"2024-03-28T01:06:22.719869+0000","last_unstale":"2024-03-28T01:08:40.027907+0000","last_undegraded":"2024-03-28T01:08:40.027907+0000","last_fullsized":"2024-03-28T01:08:40.027907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'17","last_scrub_stamp":"2024-03-28T01:08:40.027707+0000","last_deep_scrub":"23'17","last_deep_scrub_stamp":"2024-03-28T01:08:40.027707+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.027707+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:25:51.432931+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:37.443068+0000","last_change":"2024-03-28T01:08:37.443138+0000","last_active":"2024-03-28T01:08:37.443068+0000","last_peered":"2024-03-28T01:08:37.443068+0000","last_clean":"2024-03-28T01:08:37.443068+0000","last_became_active":"2024-03-28T01:06:21.723924+0000","last_became_peered":"2024-03-28T01:06:21.723924+0000","last_unstale":"2024-03-28T01:08:37.443068+0000","last_undegraded":"2024-03-28T01:08:37.443068+0000","last_fullsized":"2024-03-28T01:08:37.443068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'1","last_scrub_stamp":"2024-03-28T01:08:37.442883+0000","last_deep_scrub":"23'1","last_deep_scrub_stamp":"2024-03-28T01:08:37.442883+0000","last_clean_scrub_stamp":"2024-03-28T01:08:37.442883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:25:48.848107+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'14","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:37.659310+0000","last_change":"2024-03-28T01:08:37.659474+0000","last_active":"2024-03-28T01:08:37.659310+0000","last_peered":"2024-03-28T01:08:37.659310+0000","last_clean":"2024-03-28T01:08:37.659310+0000","last_became_active":"2024-03-28T01:06:22.717830+0000","last_became_peered":"2024-03-28T01:06:22.717830+0000","last_unstale":"2024-03-28T01:08:37.659310+0000","last_undegraded":"2024-03-28T01:08:37.659310+0000","last_fullsized":"2024-03-28T01:08:37.659310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:37.659080+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:37.659080+0000","last_clean_scrub_stamp":"2024-03-28T01:08:37.659080+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:30:19.299994+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:36.044693+0000","last_change":"2024-03-28T01:08:36.044733+0000","last_active":"2024-03-28T01:08:36.044693+0000","last_peered":"2024-03-28T01:08:36.044693+0000","last_clean":"2024-03-28T01:08:36.044693+0000","last_became_active":"2024-03-28T01:06:21.717762+0000","last_became_peered":"2024-03-28T01:06:21.717762+0000","last_unstale":"2024-03-28T01:08:36.044693+0000","last_undegraded":"2024-03-28T01:08:36.044693+0000","last_fullsized":"2024-03-28T01:08:36.044693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:36.044555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:36.044555+0000","last_clean_scrub_stamp":"2024-03-28T01:08:36.044555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:59:02.581458+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:45.424485+0000","last_change":"2024-03-28T01:08:45.424548+0000","last_active":"2024-03-28T01:08:45.424485+0000","last_peered":"2024-03-28T01:08:45.424485+0000","last_clean":"2024-03-28T01:08:45.424485+0000","last_became_active":"2024-03-28T01:06:22.745027+0000","last_became_peered":"2024-03-28T01:06:22.745027+0000","last_unstale":"2024-03-28T01:08:45.424485+0000","last_undegraded":"2024-03-28T01:08:45.424485+0000","last_fullsized":"2024-03-28T01:08:45.424485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:45.424201+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:45.424201+0000","last_clean_scrub_stamp":"2024-03-28T01:08:45.424201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:30:54.306625+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:43.831262+0000","last_change":"2024-03-28T01:08:43.831517+0000","last_active":"2024-03-28T01:08:43.831262+0000","last_peered":"2024-03-28T01:08:43.831262+0000","last_clean":"2024-03-28T01:08:43.831262+0000","last_became_active":"2024-03-28T01:06:21.734628+0000","last_became_peered":"2024-03-28T01:06:21.734628+0000","last_unstale":"2024-03-28T01:08:43.831262+0000","last_undegraded":"2024-03-28T01:08:43.831262+0000","last_fullsized":"2024-03-28T01:08:43.831262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:43.831045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:43.831045+0000","last_clean_scrub_stamp":"2024-03-28T01:08:43.831045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:42:10.729157+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'18","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:41.012342+0000","last_change":"2024-03-28T01:08:41.012403+0000","last_active":"2024-03-28T01:08:41.012342+0000","last_peered":"2024-03-28T01:08:41.012342+0000","last_clean":"2024-03-28T01:08:41.012342+0000","last_became_active":"2024-03-28T01:06:22.722140+0000","last_became_peered":"2024-03-28T01:06:22.722140+0000","last_unstale":"2024-03-28T01:08:41.012342+0000","last_undegraded":"2024-03-28T01:08:41.012342+0000","last_fullsized":"2024-03-28T01:08:41.012342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'18","last_scrub_stamp":"2024-03-28T01:08:41.012142+0000","last_deep_scrub":"23'18","last_deep_scrub_stamp":"2024-03-28T01:08:41.012142+0000","last_clean_scrub_stamp":"2024-03-28T01:08:41.012142+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:34:00.805698+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'6","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:38.061092+0000","last_change":"2024-03-28T01:08:38.061217+0000","last_active":"2024-03-28T01:08:38.061092+0000","last_peered":"2024-03-28T01:08:38.061092+0000","last_clean":"2024-03-28T01:08:38.061092+0000","last_became_active":"2024-03-28T01:06:21.723810+0000","last_became_peered":"2024-03-28T01:06:21.723810+0000","last_unstale":"2024-03-28T01:08:38.061092+0000","last_undegraded":"2024-03-28T01:08:38.061092+0000","last_fullsized":"2024-03-28T01:08:38.061092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-03-28T01:08:38.060865+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-03-28T01:08:38.060865+0000","last_clean_scrub_stamp":"2024-03-28T01:08:38.060865+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:39:55.721430+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'13","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:48.129746+0000","last_change":"2024-03-28T01:08:48.129809+0000","last_active":"2024-03-28T01:08:48.129746+0000","last_peered":"2024-03-28T01:08:48.129746+0000","last_clean":"2024-03-28T01:08:48.129746+0000","last_became_active":"2024-03-28T01:06:22.748086+0000","last_became_peered":"2024-03-28T01:06:22.748086+0000","last_unstale":"2024-03-28T01:08:48.129746+0000","last_undegraded":"2024-03-28T01:08:48.129746+0000","last_fullsized":"2024-03-28T01:08:48.129746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13","last_scrub_stamp":"2024-03-28T01:08:48.129562+0000","last_deep_scrub":"23'13","last_deep_scrub_stamp":"2024-03-28T01:08:48.129562+0000","last_clean_scrub_stamp":"2024-03-28T01:08:48.129562+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:21:23.548889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'3","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:41.712378+0000","last_change":"2024-03-28T01:08:41.712480+0000","last_active":"2024-03-28T01:08:41.712378+0000","last_peered":"2024-03-28T01:08:41.712378+0000","last_clean":"2024-03-28T01:08:41.712378+0000","last_became_active":"2024-03-28T01:06:21.736934+0000","last_became_peered":"2024-03-28T01:06:21.736934+0000","last_unstale":"2024-03-28T01:08:41.712378+0000","last_undegraded":"2024-03-28T01:08:41.712378+0000","last_fullsized":"2024-03-28T01:08:41.712378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-03-28T01:08:41.712199+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-03-28T01:08:41.712199+0000","last_clean_scrub_stamp":"2024-03-28T01:08:41.712199+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:39:59.372764+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'14","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:38.208525+0000","last_change":"2024-03-28T01:08:38.208576+0000","last_active":"2024-03-28T01:08:38.208525+0000","last_peered":"2024-03-28T01:08:38.208525+0000","last_clean":"2024-03-28T01:08:38.208525+0000","last_became_active":"2024-03-28T01:06:22.725332+0000","last_became_peered":"2024-03-28T01:06:22.725332+0000","last_unstale":"2024-03-28T01:08:38.208525+0000","last_undegraded":"2024-03-28T01:08:38.208525+0000","last_fullsized":"2024-03-28T01:08:38.208525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-03-28T01:08:38.208392+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-03-28T01:08:38.208392+0000","last_clean_scrub_stamp":"2024-03-28T01:08:38.208392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:25:49.613616+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:41.873800+0000","last_change":"2024-03-28T01:08:41.873851+0000","last_active":"2024-03-28T01:08:41.873800+0000","last_peered":"2024-03-28T01:08:41.873800+0000","last_clean":"2024-03-28T01:08:41.873800+0000","last_became_active":"2024-03-28T01:06:21.726497+0000","last_became_peered":"2024-03-28T01:06:21.726497+0000","last_unstale":"2024-03-28T01:08:41.873800+0000","last_undegraded":"2024-03-28T01:08:41.873800+0000","last_fullsized":"2024-03-28T01:08:41.873800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:41.873606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:41.873606+0000","last_clean_scrub_stamp":"2024-03-28T01:08:41.873606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:36:53.361331+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'3","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:36.609000+0000","last_change":"2024-03-28T01:08:36.609128+0000","last_active":"2024-03-28T01:08:36.609000+0000","last_peered":"2024-03-28T01:08:36.609000+0000","last_clean":"2024-03-28T01:08:36.609000+0000","last_became_active":"2024-03-28T01:06:22.717391+0000","last_became_peered":"2024-03-28T01:06:22.717391+0000","last_unstale":"2024-03-28T01:08:36.609000+0000","last_undegraded":"2024-03-28T01:08:36.609000+0000","last_fullsized":"2024-03-28T01:08:36.609000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-03-28T01:08:36.608773+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-03-28T01:08:36.608773+0000","last_clean_scrub_stamp":"2024-03-28T01:08:36.608773+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:36:07.375066+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:39.054063+0000","last_change":"2024-03-28T01:08:39.054115+0000","last_active":"2024-03-28T01:08:39.054063+0000","last_peered":"2024-03-28T01:08:39.054063+0000","last_clean":"2024-03-28T01:08:39.054063+0000","last_became_active":"2024-03-28T01:06:21.727935+0000","last_became_peered":"2024-03-28T01:06:21.727935+0000","last_unstale":"2024-03-28T01:08:39.054063+0000","last_undegraded":"2024-03-28T01:08:39.054063+0000","last_fullsized":"2024-03-28T01:08:39.054063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:39.053847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:39.053847+0000","last_clean_scrub_stamp":"2024-03-28T01:08:39.053847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:02:38.863056+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'18","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:37.980440+0000","last_change":"2024-03-28T01:08:37.980499+0000","last_active":"2024-03-28T01:08:37.980440+0000","last_peered":"2024-03-28T01:08:37.980440+0000","last_clean":"2024-03-28T01:08:37.980440+0000","last_became_active":"2024-03-28T01:06:22.723748+0000","last_became_peered":"2024-03-28T01:06:22.723748+0000","last_unstale":"2024-03-28T01:08:37.980440+0000","last_undegraded":"2024-03-28T01:08:37.980440+0000","last_fullsized":"2024-03-28T01:08:37.980440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'18","last_scrub_stamp":"2024-03-28T01:08:37.980260+0000","last_deep_scrub":"23'18","last_deep_scrub_stamp":"2024-03-28T01:08:37.980260+0000","last_clean_scrub_stamp":"2024-03-28T01:08:37.980260+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:10:43.391056+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:34.005454+0000","last_change":"2024-03-28T01:08:34.005454+0000","last_active":"2024-03-28T01:08:34.005454+0000","last_peered":"2024-03-28T01:08:34.005454+0000","last_clean":"2024-03-28T01:08:34.005454+0000","last_became_active":"2024-03-28T01:06:21.714011+0000","last_became_peered":"2024-03-28T01:06:21.714011+0000","last_unstale":"2024-03-28T01:08:34.005454+0000","last_undegraded":"2024-03-28T01:08:34.005454+0000","last_fullsized":"2024-03-28T01:08:34.005454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T01:08:34.005348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T01:08:34.005348+0000","last_clean_scrub_stamp":"2024-03-28T01:08:34.005348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:18:05.676086+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'16","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:46.110354+0000","last_change":"2024-03-28T01:08:46.110466+0000","last_active":"2024-03-28T01:08:46.110354+0000","last_peered":"2024-03-28T01:08:46.110354+0000","last_clean":"2024-03-28T01:08:46.110354+0000","last_became_active":"2024-03-28T01:06:22.738814+0000","last_became_peered":"2024-03-28T01:06:22.738814+0000","last_unstale":"2024-03-28T01:08:46.110354+0000","last_undegraded":"2024-03-28T01:08:46.110354+0000","last_fullsized":"2024-03-28T01:08:46.110354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'16","last_scrub_stamp":"2024-03-28T01:08:46.110036+0000","last_deep_scrub":"23'16","last_deep_scrub_stamp":"2024-03-28T01:08:46.110036+0000","last_clean_scrub_stamp":"2024-03-28T01:08:46.110036+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:52:29.504877+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'4","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:44.823738+0000","last_change":"2024-03-28T01:08:44.823983+0000","last_active":"2024-03-28T01:08:44.823738+0000","last_peered":"2024-03-28T01:08:44.823738+0000","last_clean":"2024-03-28T01:08:44.823738+0000","last_became_active":"2024-03-28T01:06:21.735003+0000","last_became_peered":"2024-03-28T01:06:21.735003+0000","last_unstale":"2024-03-28T01:08:44.823738+0000","last_undegraded":"2024-03-28T01:08:44.823738+0000","last_fullsized":"2024-03-28T01:08:44.823738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-03-28T01:08:44.823465+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-03-28T01:08:44.823465+0000","last_clean_scrub_stamp":"2024-03-28T01:08:44.823465+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:28:46.108625+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:46.760472+0000","last_change":"2024-03-28T01:08:46.760537+0000","last_active":"2024-03-28T01:08:46.760472+0000","last_peered":"2024-03-28T01:08:46.760472+0000","last_clean":"2024-03-28T01:08:46.760472+0000","last_became_active":"2024-03-28T01:06:22.729441+0000","last_became_peered":"2024-03-28T01:06:22.729441+0000","last_unstale":"2024-03-28T01:08:46.760472+0000","last_undegraded":"2024-03-28T01:08:46.760472+0000","last_fullsized":"2024-03-28T01:08:46.760472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:46.760163+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:46.760163+0000","last_clean_scrub_stamp":"2024-03-28T01:08:46.760163+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:16:58.139405+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'29","reported_seq":57,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:44.414887+0000","last_change":"2024-03-28T01:08:44.414887+0000","last_active":"2024-03-28T01:08:44.414887+0000","last_peered":"2024-03-28T01:08:44.414887+0000","last_clean":"2024-03-28T01:08:44.414887+0000","last_became_active":"2024-03-28T01:06:22.737509+0000","last_became_peered":"2024-03-28T01:06:22.737509+0000","last_unstale":"2024-03-28T01:08:44.414887+0000","last_undegraded":"2024-03-28T01:08:44.414887+0000","last_fullsized":"2024-03-28T01:08:44.414887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'29","last_scrub_stamp":"2024-03-28T01:08:44.414639+0000","last_deep_scrub":"23'29","last_deep_scrub_stamp":"2024-03-28T01:08:44.414639+0000","last_clean_scrub_stamp":"2024-03-28T01:08:44.414639+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:52:27.809480+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'3","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:42.867996+0000","last_change":"2024-03-28T01:08:42.867996+0000","last_active":"2024-03-28T01:08:42.867996+0000","last_peered":"2024-03-28T01:08:42.867996+0000","last_clean":"2024-03-28T01:08:42.867996+0000","last_became_active":"2024-03-28T01:06:21.734646+0000","last_became_peered":"2024-03-28T01:06:21.734646+0000","last_unstale":"2024-03-28T01:08:42.867996+0000","last_undegraded":"2024-03-28T01:08:42.867996+0000","last_fullsized":"2024-03-28T01:08:42.867996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-03-28T01:08:42.867749+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-03-28T01:08:42.867749+0000","last_clean_scrub_stamp":"2024-03-28T01:08:42.867749+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:16:54.246990+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'18","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:41.409716+0000","last_change":"2024-03-28T01:08:41.409716+0000","last_active":"2024-03-28T01:08:41.409716+0000","last_peered":"2024-03-28T01:08:41.409716+0000","last_clean":"2024-03-28T01:08:41.409716+0000","last_became_active":"2024-03-28T01:06:22.730668+0000","last_became_peered":"2024-03-28T01:06:22.730668+0000","last_unstale":"2024-03-28T01:08:41.409716+0000","last_undegraded":"2024-03-28T01:08:41.409716+0000","last_fullsized":"2024-03-28T01:08:41.409716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'18","last_scrub_stamp":"2024-03-28T01:08:41.409461+0000","last_deep_scrub":"23'18","last_deep_scrub_stamp":"2024-03-28T01:08:41.409461+0000","last_clean_scrub_stamp":"2024-03-28T01:08:41.409461+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:28:42.694620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'4","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:40.902982+0000","last_change":"2024-03-28T01:08:40.902982+0000","last_active":"2024-03-28T01:08:40.902982+0000","last_peered":"2024-03-28T01:08:40.902982+0000","last_clean":"2024-03-28T01:08:40.902982+0000","last_became_active":"2024-03-28T01:06:21.730801+0000","last_became_peered":"2024-03-28T01:06:21.730801+0000","last_unstale":"2024-03-28T01:08:40.902982+0000","last_undegraded":"2024-03-28T01:08:40.902982+0000","last_fullsized":"2024-03-28T01:08:40.902982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-03-28T01:08:40.902718+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-03-28T01:08:40.902718+0000","last_clean_scrub_stamp":"2024-03-28T01:08:40.902718+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:41:12.282876+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'12","reported_seq":45,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:43.231774+0000","last_change":"2024-03-28T01:08:43.231841+0000","last_active":"2024-03-28T01:08:43.231774+0000","last_peered":"2024-03-28T01:08:43.231774+0000","last_clean":"2024-03-28T01:08:43.231774+0000","last_became_active":"2024-03-28T01:07:03.941880+0000","last_became_peered":"2024-03-28T01:07:03.941880+0000","last_unstale":"2024-03-28T01:08:43.231774+0000","last_undegraded":"2024-03-28T01:08:43.231774+0000","last_fullsized":"2024-03-28T01:08:43.231774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-03-28T01:08:43.231528+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-03-28T01:08:43.231528+0000","last_clean_scrub_stamp":"2024-03-28T01:08:43.231528+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:01:39.374318+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:35.913509+0000","last_change":"2024-03-28T01:08:35.913558+0000","last_active":"2024-03-28T01:08:35.913509+0000","last_peered":"2024-03-28T01:08:35.913509+0000","last_clean":"2024-03-28T01:08:35.913509+0000","last_became_active":"2024-03-28T01:06:21.722751+0000","last_became_peered":"2024-03-28T01:06:21.722751+0000","last_unstale":"2024-03-28T01:08:35.913509+0000","last_undegraded":"2024-03-28T01:08:35.913509+0000","last_fullsized":"2024-03-28T01:08:35.913509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2024-03-28T01:08:35.913312+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2024-03-28T01:08:35.913312+0000","last_clean_scrub_stamp":"2024-03-28T01:08:35.913312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:21:16.986850+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:44.065093+0000","last_change":"2024-03-28T01:08:44.065155+0000","last_active":"2024-03-28T01:08:44.065093+0000","last_peered":"2024-03-28T01:08:44.065093+0000","last_clean":"2024-03-28T01:08:44.065093+0000","last_became_active":"2024-03-28T01:06:22.739270+0000","last_became_peered":"2024-03-28T01:06:22.739270+0000","last_unstale":"2024-03-28T01:08:44.065093+0000","last_undegraded":"2024-03-28T01:08:44.065093+0000","last_fullsized":"2024-03-28T01:08:44.065093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-03-28T01:08:44.064901+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-03-28T01:08:44.064901+0000","last_clean_scrub_stamp":"2024-03-28T01:08:44.064901+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:01:40.207691+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T01:08:33.245717+0000","last_change":"2024-03-28T01:08:33.245788+0000","last_active":"2024-03-28T01:08:33.245717+0000","last_peered":"2024-03-28T01:08:33.245717+0000","last_clean":"2024-03-28T01:08:33.245717+0000","last_became_active":"2024-03-28T01:06:21.727407+0000","last_became_peered":"2024-03-28T01:06:21.727407+0000","last_unstale":"2024-03-28T01:08:33.245717+0000","last_undegraded":"2024-03-28T01:08:33.245717+0000","last_fullsized":"2024-03-28T01:08:33.245717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-03-28T01:08:33.245481+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-03-28T01:08:33.245481+0000","last_clean_scrub_stamp":"2024-03-28T01:08:33.245481+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:56:41.786549+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":21,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":42,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1028,"ondisk_log_size":1028,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":150794,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182,"num_write_kb":239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":301588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":856064,"data_stored":680429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":401408,"data_stored":238352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":89563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":89596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":421888,"data_stored":239475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":90584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":89627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":851968,"data_stored":680476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":149996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":148885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":6,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":6,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T01:08:58.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-28T01:08:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.219+0000 7f5790800640 1 Processor -- start 2024-03-28T01:08:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.219+0000 7f5790800640 1 -- start start 2024-03-28T01:08:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.221+0000 7f5790800640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f5788144f80 0x7f5788152d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.221+0000 7f5790800640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578814b160 con 0x7f57880b3bd0 2024-03-28T01:08:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.221+0000 7f5790800640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57880b4330 con 0x7f5788067040 2024-03-28T01:08:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.221+0000 7f5790800640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57880ae560 con 0x7f5788144f80 2024-03-28T01:08:59.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578d400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f57880b3bd0 0x7f5788144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37344/0 (socket says 172.21.15.33:37344) 2024-03-28T01:08:59.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578d400640 1 -- 172.21.15.33:0/2002698836 learned_addr learned my addr 172.21.15.33:0/2002698836 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:59.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578e800640 1 --2- 172.21.15.33:0/2002698836 >> v2:172.21.15.194:3300/0 conn(0x7f5788144f80 0x7f5788152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578ca00640 1 -- 172.21.15.33:0/2002698836 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86831467 0 0) 0x7f578814b160 con 0x7f57880b3bd0 2024-03-28T01:08:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578ca00640 1 -- 172.21.15.33:0/2002698836 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5770003520 con 0x7f57880b3bd0 2024-03-28T01:08:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578ca00640 1 -- 172.21.15.33:0/2002698836 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542638326 0 0) 0x7f5770003520 con 0x7f57880b3bd0 2024-03-28T01:08:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578e800640 1 -- 172.21.15.33:0/2002698836 >> v1:172.21.15.194:6789/0 conn(0x7f5788067040 legacy=0x7f578814aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.222+0000 7f578e800640 1 -- 172.21.15.33:0/2002698836 >> v1:172.21.15.33:6789/0 conn(0x7f57880b3bd0 legacy=0x7f5788144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.223+0000 7f578e800640 1 -- 172.21.15.33:0/2002698836 --> v2:172.21.15.194:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57880b4b20 con 0x7f5788144f80 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.224+0000 7f578e800640 1 --2- 172.21.15.33:0/2002698836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788152d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5784009d90 tx=0x7f578402bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4def1fe064706eef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.224+0000 7f578ca00640 1 -- 172.21.15.33:0/2002698836 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784039070 con 0x7f5788144f80 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.224+0000 7f578ca00640 1 -- 172.21.15.33:0/2002698836 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578402c820 con 0x7f5788144f80 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.224+0000 7f578ca00640 1 -- 172.21.15.33:0/2002698836 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778001670 con 0x7f5788144f80 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.225+0000 7f5790800640 1 -- 172.21.15.33:0/2002698836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 msgr2=0x7f5788152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.225+0000 7f5790800640 1 --2- 172.21.15.33:0/2002698836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788152d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5784009d90 tx=0x7f578402bcf0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.225+0000 7f5790800640 1 -- 172.21.15.33:0/2002698836 shutdown_connections 2024-03-28T01:08:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.225+0000 7f5790800640 1 --2- 172.21.15.33:0/2002698836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788152d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.225+0000 7f5790800640 1 -- 172.21.15.33:0/2002698836 >> 172.21.15.33:0/2002698836 conn(0x7f57880b9800 msgr2=0x7f57880b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.225+0000 7f5790800640 1 -- 172.21.15.33:0/2002698836 shutdown_connections 2024-03-28T01:08:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.225+0000 7f5790800640 1 -- 172.21.15.33:0/2002698836 wait complete. 2024-03-28T01:08:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.227+0000 7f5790800640 1 Processor -- start 2024-03-28T01:08:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.228+0000 7f5790800640 1 -- start start 2024-03-28T01:08:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f5790800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5788067040 0x7f578806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f5790800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57880b3bd0 0x7f57881dcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f5790800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f5790800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57880b4dc0 con 0x7f57880b3bd0 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f5790800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57880b6b40 con 0x7f5788144f80 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f5790800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57880ae7b0 con 0x7f5788067040 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f578d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57880b3bd0 0x7f57881dcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f578d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57880b3bd0 0x7f57881dcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36196/0 (socket says 172.21.15.33:36196) 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.229+0000 7f578d400640 1 -- 172.21.15.33:0/1507031174 learned_addr learned my addr 172.21.15.33:0/1507031174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578d400640 1 -- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5788067040 msgr2=0x7f578806fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578e800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578de00640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5788067040 0x7f578806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578d400640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5788067040 0x7f578806fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578d400640 1 -- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 msgr2=0x7f5788147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578d400640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578d400640 1 -- 172.21.15.33:0/1507031174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788071fa0 con 0x7f57880b3bd0 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578e800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578de00640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5788067040 0x7f578806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f578d400640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57880b3bd0 0x7f57881dcef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f577800c730 tx=0x7f577800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.230+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778017070 con 0x7f57880b3bd0 2024-03-28T01:08:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.231+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778005af0 con 0x7f57880b3bd0 2024-03-28T01:08:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.231+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778007070 con 0x7f57880b3bd0 2024-03-28T01:08:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.231+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5788072710 con 0x7f57880b3bd0 2024-03-28T01:08:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.231+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57881dd430 con 0x7f57880b3bd0 2024-03-28T01:08:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.232+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57880b56a0 con 0x7f57880b3bd0 2024-03-28T01:08:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.233+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f577801a070 con 0x7f57880b3bd0 2024-03-28T01:08:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.234+0000 7f5782000640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f576c06bbd0 0x7f576c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.234+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f577808a150 con 0x7f57880b3bd0 2024-03-28T01:08:59.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.234+0000 7f578de00640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f576c06bbd0 0x7f576c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.235+0000 7f578de00640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f576c06bbd0 0x7f576c06e090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f60 tx=0x7f577c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:59.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.239+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f57880b56a0 con 0x7f57880b3bd0 2024-03-28T01:08:59.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.438+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f5788075380 con 0x7f57880b3bd0 2024-03-28T01:08:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.444+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5778058ad0 con 0x7f57880b3bd0 2024-03-28T01:08:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.445+0000 7f5782000640 1 -- 172.21.15.33:0/1507031174 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f57780583d0 con 0x7f57880b3bd0 2024-03-28T01:08:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f576c06bbd0 msgr2=0x7f576c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f576c06bbd0 0x7f576c06e090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f60 tx=0x7f577c009040 comp rx=0 tx=0).stop 2024-03-28T01:08:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57880b3bd0 msgr2=0x7f57881dcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57880b3bd0 0x7f57881dcef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f577800c730 tx=0x7f577800cbf0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 shutdown_connections 2024-03-28T01:08:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f576c06bbd0 0x7f576c06e090 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788144f80 0x7f5788147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57880b3bd0 0x7f57881dcef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 --2- 172.21.15.33:0/1507031174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5788067040 0x7f578806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 >> 172.21.15.33:0/1507031174 conn(0x7f57880b9800 msgr2=0x7f5788144640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.447+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 shutdown_connections 2024-03-28T01:08:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.448+0000 7f5790800640 1 -- 172.21.15.33:0/1507031174 wait complete. 2024-03-28T01:08:59.478 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T01:08:59.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T01:08:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.690+0000 7f9dc9200640 1 Processor -- start 2024-03-28T01:08:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.690+0000 7f9dc9200640 1 -- start start 2024-03-28T01:08:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.691+0000 7f9dc9200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9dc40b8d00 0x7f9dc414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.691+0000 7f9dc9200640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40b55b0 con 0x7f9dc40b51d0 2024-03-28T01:08:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.691+0000 7f9dc9200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40b9190 con 0x7f9dc40b6b80 2024-03-28T01:08:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.691+0000 7f9dc9200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc414a820 con 0x7f9dc40b8d00 2024-03-28T01:08:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc3400640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9dc40b51d0 0x7f9dc40b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37350/0 (socket says 172.21.15.33:37350) 2024-03-28T01:08:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc3400640 1 -- 172.21.15.33:0/3038580175 learned_addr learned my addr 172.21.15.33:0/3038580175 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:08:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc2a00640 1 --2- 172.21.15.33:0/3038580175 >> v2:172.21.15.33:3300/0 conn(0x7f9dc40b8d00 0x7f9dc414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc2000640 1 -- 172.21.15.33:0/3038580175 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271062705 0 0) 0x7f9dc40b55b0 con 0x7f9dc40b51d0 2024-03-28T01:08:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc2000640 1 -- 172.21.15.33:0/3038580175 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da8003520 con 0x7f9dc40b51d0 2024-03-28T01:08:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc2a00640 1 -- 172.21.15.33:0/3038580175 >> v1:172.21.15.194:6789/0 conn(0x7f9dc40b6b80 legacy=0x7f9dc414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:08:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc2a00640 1 -- 172.21.15.33:0/3038580175 >> v1:172.21.15.33:6789/0 conn(0x7f9dc40b51d0 legacy=0x7f9dc40b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.692+0000 7f9dc2a00640 1 -- 172.21.15.33:0/3038580175 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc40b9800 con 0x7f9dc40b8d00 2024-03-28T01:08:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.693+0000 7f9dc2a00640 1 --2- 172.21.15.33:0/3038580175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b8d00 0x7f9dc414ac70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0097a0 tx=0x7f9dac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50a0572f5b23322 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc2000640 1 -- 172.21.15.33:0/3038580175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac035020 con 0x7f9dc40b8d00 2024-03-28T01:08:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc2000640 1 -- 172.21.15.33:0/3038580175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9dac031830 con 0x7f9dc40b8d00 2024-03-28T01:08:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc2000640 1 -- 172.21.15.33:0/3038580175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac031be0 con 0x7f9dc40b8d00 2024-03-28T01:08:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc9200640 1 -- 172.21.15.33:0/3038580175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b8d00 msgr2=0x7f9dc414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc9200640 1 --2- 172.21.15.33:0/3038580175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b8d00 0x7f9dc414ac70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0097a0 tx=0x7f9dac02b8d0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc9200640 1 -- 172.21.15.33:0/3038580175 shutdown_connections 2024-03-28T01:08:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc9200640 1 --2- 172.21.15.33:0/3038580175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b8d00 0x7f9dc414ac70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.694+0000 7f9dc9200640 1 -- 172.21.15.33:0/3038580175 >> 172.21.15.33:0/3038580175 conn(0x7f9dc4072940 msgr2=0x7f9dc4072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:59.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.695+0000 7f9dc9200640 1 -- 172.21.15.33:0/3038580175 shutdown_connections 2024-03-28T01:08:59.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.695+0000 7f9dc9200640 1 -- 172.21.15.33:0/3038580175 wait complete. 2024-03-28T01:08:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.696+0000 7f9dc9200640 1 Processor -- start 2024-03-28T01:08:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.696+0000 7f9dc9200640 1 -- start start 2024-03-28T01:08:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc9200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b51d0 0x7f9dc413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc9200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dc40b6b80 0x7f9dc41e0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc9200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9dc4144200 0x7f9dc4140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc9200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40b39c0 con 0x7f9dc40b51d0 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc9200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40b44c0 con 0x7f9dc40b6b80 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc9200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4152b80 con 0x7f9dc4144200 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b51d0 0x7f9dc413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b51d0 0x7f9dc413d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36222/0 (socket says 172.21.15.33:36222) 2024-03-28T01:08:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc3400640 1 -- 172.21.15.33:0/2317245639 learned_addr learned my addr 172.21.15.33:0/2317245639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc2a00640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dc40b6b80 0x7f9dc41e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.697+0000 7f9dc3e00640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9dc4144200 0x7f9dc4140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc3400640 1 -- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9dc4144200 msgr2=0x7f9dc4140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc3400640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9dc4144200 0x7f9dc4140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc3400640 1 -- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dc40b6b80 msgr2=0x7f9dc41e0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc3400640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dc40b6b80 0x7f9dc41e0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc3400640 1 -- 172.21.15.33:0/2317245639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc413e8c0 con 0x7f9dc40b51d0 2024-03-28T01:08:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc3e00640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9dc4144200 0x7f9dc4140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:08:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc3400640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b51d0 0x7f9dc413d0b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c430 tx=0x7f9db400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dbb400640 1 -- 172.21.15.33:0/2317245639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4017070 con 0x7f9dc40b51d0 2024-03-28T01:08:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dbb400640 1 -- 172.21.15.33:0/2317245639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9db4005ce0 con 0x7f9dc40b51d0 2024-03-28T01:08:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc40b3db0 con 0x7f9dc40b51d0 2024-03-28T01:08:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dbb400640 1 -- 172.21.15.33:0/2317245639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4007310 con 0x7f9dc40b51d0 2024-03-28T01:08:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.698+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc4144c60 con 0x7f9dc40b51d0 2024-03-28T01:08:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.699+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc4075310 con 0x7f9dc40b51d0 2024-03-28T01:08:59.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.701+0000 7f9dbb400640 1 -- 172.21.15.33:0/2317245639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9db401a070 con 0x7f9dc40b51d0 2024-03-28T01:08:59.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.701+0000 7f9dbb400640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9da006bbd0 0x7f9da006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:08:59.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.701+0000 7f9dbb400640 1 -- 172.21.15.33:0/2317245639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f9db408a0e0 con 0x7f9dc40b51d0 2024-03-28T01:08:59.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.702+0000 7f9dc2a00640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9da006bbd0 0x7f9da006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:08:59.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.703+0000 7f9dc2a00640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9da006bbd0 0x7f9da006e090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9dac036000 tx=0x7f9dac03c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:08:59.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.706+0000 7f9dbb400640 1 -- 172.21.15.33:0/2317245639 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9db4058a50 con 0x7f9dc40b51d0 2024-03-28T01:08:59.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.953+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc4146360 con 0x7f9dc40b51d0 2024-03-28T01:08:59.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.954+0000 7f9dbb400640 1 -- 172.21.15.33:0/2317245639 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+3391 (secure 0 0 0) 0x7f9db4058350 con 0x7f9dc40b51d0 2024-03-28T01:08:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:08:59.958 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/214301400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":214301400},{"type":"v1","addr":"172.21.15.194:6833","nonce":214301400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T01:06:22.173766+0000","modified":"2024-03-28T01:06:25.239216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2290842450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2290842450},{"type":"v1","addr":"172.21.15.33:6835","nonce":2290842450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-03-28T01:08:59.958 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-03-28T01:08:59.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.956+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9da006bbd0 msgr2=0x7f9da006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.956+0000 7f9dc9200640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9da006bbd0 0x7f9da006e090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9dac036000 tx=0x7f9dac03c040 comp rx=0 tx=0).stop 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.956+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b51d0 msgr2=0x7f9dc413d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.956+0000 7f9dc9200640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b51d0 0x7f9dc413d0b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c430 tx=0x7f9db400c8f0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 shutdown_connections 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f9da006bbd0 0x7f9da006e090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9dc4144200 0x7f9dc4140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dc40b6b80 0x7f9dc41e0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 --2- 172.21.15.33:0/2317245639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc40b51d0 0x7f9dc413d0b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:08:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 >> 172.21.15.33:0/2317245639 conn(0x7f9dc4072940 msgr2=0x7f9dc4148620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:08:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 shutdown_connections 2024-03-28T01:08:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:08:59.957+0000 7f9dc9200640 1 -- 172.21.15.33:0/2317245639 wait complete. 2024-03-28T01:08:59.981 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T01:09:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.174+0000 7f71ab200640 1 Processor -- start 2024-03-28T01:09:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.174+0000 7f71ab200640 1 -- start start 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71ab200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac13e360 0x7f71ac143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71ab200640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71ac06b870 0x7f71ac1443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71ab200640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac0b4910 con 0x7f71ac0b4530 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71ab200640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0b1730 con 0x7f71ac06b870 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71ab200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0b1a10 con 0x7f71ac13e360 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71aa800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71ac06b870 0x7f71ac1443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71aa800640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71ac06b870 0x7f71ac1443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36226/0 (socket says 172.21.15.33:36226) 2024-03-28T01:09:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71aa800640 1 -- 172.21.15.33:0/2199275201 learned_addr learned my addr 172.21.15.33:0/2199275201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.177+0000 7f71a9400640 1 --2- 172.21.15.33:0/2199275201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac13e360 0x7f71ac143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.178+0000 7f71aa800640 1 -- 172.21.15.33:0/2199275201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac13e360 msgr2=0x7f71ac143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.178+0000 7f71aa800640 1 --2- 172.21.15.33:0/2199275201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac13e360 0x7f71ac143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.178+0000 7f71aa800640 1 -- 172.21.15.33:0/2199275201 >> v1:172.21.15.194:6789/0 conn(0x7f71ac0b4530 legacy=0x7f71ac13dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.178+0000 7f71aa800640 1 -- 172.21.15.33:0/2199275201 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0b1cf0 con 0x7f71ac06b870 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.178+0000 7f71a9400640 1 --2- 172.21.15.33:0/2199275201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac13e360 0x7f71ac143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.179+0000 7f71aa800640 1 --2- 172.21.15.33:0/2199275201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac06b870 0x7f71ac1443c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e50 tx=0x7f71a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf4dfbd234a79dc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.179+0000 7f71a8a00640 1 -- 172.21.15.33:0/2199275201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a402ca70 con 0x7f71ac06b870 2024-03-28T01:09:00.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.179+0000 7f71a8a00640 1 -- 172.21.15.33:0/2199275201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f71a402cc10 con 0x7f71ac06b870 2024-03-28T01:09:00.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.179+0000 7f71a8a00640 1 -- 172.21.15.33:0/2199275201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40343f0 con 0x7f71ac06b870 2024-03-28T01:09:00.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 -- 172.21.15.33:0/2199275201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac06b870 msgr2=0x7f71ac1443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 --2- 172.21.15.33:0/2199275201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac06b870 0x7f71ac1443c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e50 tx=0x7f71a402bed0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 -- 172.21.15.33:0/2199275201 shutdown_connections 2024-03-28T01:09:00.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 --2- 172.21.15.33:0/2199275201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac06b870 0x7f71ac1443c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 --2- 172.21.15.33:0/2199275201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac13e360 0x7f71ac143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 -- 172.21.15.33:0/2199275201 >> 172.21.15.33:0/2199275201 conn(0x7f71ac0b89c0 msgr2=0x7f71ac0b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:00.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 -- 172.21.15.33:0/2199275201 shutdown_connections 2024-03-28T01:09:00.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.180+0000 7f71ab200640 1 -- 172.21.15.33:0/2199275201 wait complete. 2024-03-28T01:09:00.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.183+0000 7f71ab200640 1 Processor -- start 2024-03-28T01:09:00.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.183+0000 7f71ab200640 1 -- start start 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71ab200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac06b870 0x7f71ac0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71ab200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac0b4530 0x7f71ac1dde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71ab200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac13e360 0x7f71ac072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71ab200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac14e720 con 0x7f71ac13e360 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71ab200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac072e10 con 0x7f71ac0b4530 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71ab200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac14e8a0 con 0x7f71ac06b870 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71aa800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac13e360 0x7f71ac072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.184+0000 7f71aa800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac13e360 0x7f71ac072050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36236/0 (socket says 172.21.15.33:36236) 2024-03-28T01:09:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71aa800640 1 -- 172.21.15.33:0/2319988235 learned_addr learned my addr 172.21.15.33:0/2319988235 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71a9400640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac0b4530 0x7f71ac1dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71a9e00640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac06b870 0x7f71ac0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71aa800640 1 -- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac06b870 msgr2=0x7f71ac0b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71aa800640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac06b870 0x7f71ac0b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71aa800640 1 -- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac0b4530 msgr2=0x7f71ac1dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71aa800640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac0b4530 0x7f71ac1dde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71aa800640 1 -- 172.21.15.33:0/2319988235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac12ba90 con 0x7f71ac13e360 2024-03-28T01:09:00.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.185+0000 7f71aa800640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac13e360 0x7f71ac072050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e50 tx=0x7f71a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:00.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.186+0000 7f71a2000640 1 -- 172.21.15.33:0/2319988235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4043020 con 0x7f71ac13e360 2024-03-28T01:09:00.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.186+0000 7f71a2000640 1 -- 172.21.15.33:0/2319988235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f71a403e450 con 0x7f71ac13e360 2024-03-28T01:09:00.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.186+0000 7f71a2000640 1 -- 172.21.15.33:0/2319988235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a403e860 con 0x7f71ac13e360 2024-03-28T01:09:00.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.186+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac12f8b0 con 0x7f71ac13e360 2024-03-28T01:09:00.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.186+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac12fe70 con 0x7f71ac13e360 2024-03-28T01:09:00.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.187+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac06c3e0 con 0x7f71ac13e360 2024-03-28T01:09:00.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.188+0000 7f71a2000640 1 -- 172.21.15.33:0/2319988235 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f71a4046020 con 0x7f71ac13e360 2024-03-28T01:09:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.189+0000 7f71a2000640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f718c06bc20 0x7f718c06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.189+0000 7f71a2000640 1 -- 172.21.15.33:0/2319988235 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f71a40ad540 con 0x7f71ac13e360 2024-03-28T01:09:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.189+0000 7f71a9e00640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f718c06bc20 0x7f718c06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.190+0000 7f71a9e00640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f718c06bc20 0x7f718c06e0e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f60 tx=0x7f719c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.193+0000 7f71a2000640 1 -- 172.21.15.33:0/2319988235 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f71ac06c3e0 con 0x7f71ac13e360 2024-03-28T01:09:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.437+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71ac0b2ea0 con 0x7f71ac13e360 2024-03-28T01:09:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.438+0000 7f71a2000640 1 -- 172.21.15.33:0/2319988235 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+3391 (secure 0 0 0) 0x7f71ac0b2ea0 con 0x7f71ac13e360 2024-03-28T01:09:00.442 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:09:00.442 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/214301400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":214301400},{"type":"v1","addr":"172.21.15.194:6833","nonce":214301400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T01:06:22.173766+0000","modified":"2024-03-28T01:06:25.239216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2290842450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2290842450},{"type":"v1","addr":"172.21.15.33:6835","nonce":2290842450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-03-28T01:09:00.443 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-03-28T01:09:00.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f718c06bc20 msgr2=0x7f718c06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f718c06bc20 0x7f718c06e0e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f60 tx=0x7f719c009040 comp rx=0 tx=0).stop 2024-03-28T01:09:00.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac13e360 msgr2=0x7f71ac072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac13e360 0x7f71ac072050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e50 tx=0x7f71a4004a40 comp rx=0 tx=0).stop 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 shutdown_connections 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f718c06bc20 0x7f718c06e0e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71ac13e360 0x7f71ac072050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac0b4530 0x7f71ac1dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 --2- 172.21.15.33:0/2319988235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ac06b870 0x7f71ac0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 >> 172.21.15.33:0/2319988235 conn(0x7f71ac0b89c0 msgr2=0x7f71ac0b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 shutdown_connections 2024-03-28T01:09:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.441+0000 7f71ab200640 1 -- 172.21.15.33:0/2319988235 wait complete. 2024-03-28T01:09:00.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T01:09:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.673+0000 7f7c55c00640 1 Processor -- start 2024-03-28T01:09:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.674+0000 7f7c55c00640 1 -- start start 2024-03-28T01:09:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.675+0000 7f7c55c00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f7c500b9a10 0x7f7c500b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.675+0000 7f7c55c00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c500b4c10 0x7f7c500b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c55c00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c500b64e0 con 0x7f7c500b3070 2024-03-28T01:09:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c55c00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c500b5020 con 0x7f7c500b4c10 2024-03-28T01:09:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c55c00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c500b9df0 con 0x7f7c500b9a10 2024-03-28T01:09:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c47e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c500b4c10 0x7f7c500b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c47e00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c500b4c10 0x7f7c500b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36238/0 (socket says 172.21.15.33:36238) 2024-03-28T01:09:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c47e00640 1 -- 172.21.15.33:0/2939475790 learned_addr learned my addr 172.21.15.33:0/2939475790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c4fe00640 1 --2- 172.21.15.33:0/2939475790 >> v2:172.21.15.194:3300/0 conn(0x7f7c500b9a10 0x7f7c500b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c47e00640 1 -- 172.21.15.33:0/2939475790 >> v2:172.21.15.194:3300/0 conn(0x7f7c500b9a10 msgr2=0x7f7c500b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.676+0000 7f7c47e00640 1 --2- 172.21.15.33:0/2939475790 >> v2:172.21.15.194:3300/0 conn(0x7f7c500b9a10 0x7f7c500b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.677+0000 7f7c47e00640 1 -- 172.21.15.33:0/2939475790 >> v1:172.21.15.33:6789/0 conn(0x7f7c500b3070 legacy=0x7f7c500b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.677+0000 7f7c47e00640 1 -- 172.21.15.33:0/2939475790 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500b4770 con 0x7f7c500b4c10 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.678+0000 7f7c47e00640 1 --2- 172.21.15.33:0/2939475790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500b4c10 0x7f7c500b2b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c500b8a90 tx=0x7f7c3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bae5c20d9a0b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.678+0000 7f7c4f400640 1 -- 172.21.15.33:0/2939475790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38035020 con 0x7f7c500b4c10 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.678+0000 7f7c4f400640 1 -- 172.21.15.33:0/2939475790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7c38031870 con 0x7f7c500b4c10 2024-03-28T01:09:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.678+0000 7f7c4f400640 1 -- 172.21.15.33:0/2939475790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38031c60 con 0x7f7c500b4c10 2024-03-28T01:09:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 -- 172.21.15.33:0/2939475790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500b4c10 msgr2=0x7f7c500b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 --2- 172.21.15.33:0/2939475790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500b4c10 0x7f7c500b2b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c500b8a90 tx=0x7f7c3802b910 comp rx=0 tx=0).stop 2024-03-28T01:09:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 -- 172.21.15.33:0/2939475790 shutdown_connections 2024-03-28T01:09:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 --2- 172.21.15.33:0/2939475790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500b4c10 0x7f7c500b2b30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 --2- 172.21.15.33:0/2939475790 >> v2:172.21.15.194:3300/0 conn(0x7f7c500b9a10 0x7f7c500b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 -- 172.21.15.33:0/2939475790 >> 172.21.15.33:0/2939475790 conn(0x7f7c50072940 msgr2=0x7f7c50072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 -- 172.21.15.33:0/2939475790 shutdown_connections 2024-03-28T01:09:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.679+0000 7f7c55c00640 1 -- 172.21.15.33:0/2939475790 wait complete. 2024-03-28T01:09:00.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.681+0000 7f7c55c00640 1 Processor -- start 2024-03-28T01:09:00.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.682+0000 7f7c55c00640 1 -- start start 2024-03-28T01:09:00.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c500b3070 0x7f7c500bc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c500b9a10 0x7f7c500bd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500bb0c0 0x7f7c501dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500b6be0 con 0x7f7c500bb0c0 2024-03-28T01:09:00.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500b55a0 con 0x7f7c500b3070 2024-03-28T01:09:00.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500b6d60 con 0x7f7c500b9a10 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500bb0c0 0x7f7c501dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500bb0c0 0x7f7c501dd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36242/0 (socket says 172.21.15.33:36242) 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c55200640 1 -- 172.21.15.33:0/687594129 learned_addr learned my addr 172.21.15.33:0/687594129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.683+0000 7f7c4fe00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c500b3070 0x7f7c500bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c47e00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c500b9a10 0x7f7c500bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c55200640 1 -- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c500b9a10 msgr2=0x7f7c500bd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c55200640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c500b9a10 0x7f7c500bd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c55200640 1 -- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c500b3070 msgr2=0x7f7c500bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c55200640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c500b3070 0x7f7c500bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c55200640 1 -- 172.21.15.33:0/687594129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c501dcfb0 con 0x7f7c500bb0c0 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c47e00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c500b9a10 0x7f7c500bd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c55200640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500bb0c0 0x7f7c501dd7b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800c710 tx=0x7f7c4800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c4cc00640 1 -- 172.21.15.33:0/687594129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48016020 con 0x7f7c500bb0c0 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c4cc00640 1 -- 172.21.15.33:0/687594129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7c48006d50 con 0x7f7c500bb0c0 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c4cc00640 1 -- 172.21.15.33:0/687594129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48003bd0 con 0x7f7c500bb0c0 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.684+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c500bb5b0 con 0x7f7c500bb0c0 2024-03-28T01:09:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.685+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c501d16e0 con 0x7f7c500bb0c0 2024-03-28T01:09:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.686+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c50073770 con 0x7f7c500bb0c0 2024-03-28T01:09:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.687+0000 7f7c4cc00640 1 -- 172.21.15.33:0/687594129 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7c4800a850 con 0x7f7c500bb0c0 2024-03-28T01:09:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.688+0000 7f7c4cc00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7c2c06bbd0 0x7f7c2c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.688+0000 7f7c4cc00640 1 -- 172.21.15.33:0/687594129 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f7c4808a850 con 0x7f7c500bb0c0 2024-03-28T01:09:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.688+0000 7f7c4fe00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7c2c06bbd0 0x7f7c2c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.689+0000 7f7c4fe00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7c2c06bbd0 0x7f7c2c06e090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:00.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.692+0000 7f7c4cc00640 1 -- 172.21.15.33:0/687594129 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f7c50073770 con 0x7f7c500bb0c0 2024-03-28T01:09:00.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.897+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c50075310 con 0x7f7c500bb0c0 2024-03-28T01:09:00.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.898+0000 7f7c4cc00640 1 -- 172.21.15.33:0/687594129 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f7c48020020 con 0x7f7c500bb0c0 2024-03-28T01:09:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-03-28T01:09:00.903 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"6560509b-7594-459b-89e1-d55d3bb3e1bc","created":"2024-03-28T01:05:57.162283+0000","modified":"2024-03-28T01:08:19.131295+0000","last_up_change":"2024-03-28T01:06:09.704222+0000","last_in_change":"2024-03-28T01:06:00.987503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T01:06:10.462126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T01:06:20.177809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T01:06:21.198872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2867531b-5f43-4ce2-82ef-35bfdfd9af3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6820","nonce":1178911672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6825","nonce":1178911672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6831","nonce":1178911672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1178911672},{"type":"v1","addr":"172.21.15.33:6829","nonce":1178911672}]},"public_addr":"172.21.15.33:6820/1178911672","cluster_addr":"172.21.15.33:6825/1178911672","heartbeat_back_addr":"172.21.15.33:6831/1178911672","heartbeat_front_addr":"172.21.15.33:6829/1178911672","state":["exists","up"]},{"osd":1,"uuid":"e5d41e15-7330-4018-9ba8-6972f2bf9eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6818","nonce":4011280304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6821","nonce":4011280304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6828","nonce":4011280304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":4011280304},{"type":"v1","addr":"172.21.15.33:6824","nonce":4011280304}]},"public_addr":"172.21.15.33:6818/4011280304","cluster_addr":"172.21.15.33:6821/4011280304","heartbeat_back_addr":"172.21.15.33:6828/4011280304","heartbeat_front_addr":"172.21.15.33:6824/4011280304","state":["exists","up"]},{"osd":2,"uuid":"db38d592-5f5e-41a4-96e3-ff85f508d242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6809","nonce":1101919847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6811","nonce":1101919847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6815","nonce":1101919847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1101919847},{"type":"v1","addr":"172.21.15.33:6813","nonce":1101919847}]},"public_addr":"172.21.15.33:6809/1101919847","cluster_addr":"172.21.15.33:6811/1101919847","heartbeat_back_addr":"172.21.15.33:6815/1101919847","heartbeat_front_addr":"172.21.15.33:6813/1101919847","state":["exists","up"]},{"osd":3,"uuid":"68f5578e-d4a0-48fb-8719-b6436beb25ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6801","nonce":2717396394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6803","nonce":2717396394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6807","nonce":2717396394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2717396394},{"type":"v1","addr":"172.21.15.33:6805","nonce":2717396394}]},"public_addr":"172.21.15.33:6801/2717396394","cluster_addr":"172.21.15.33:6803/2717396394","heartbeat_back_addr":"172.21.15.33:6807/2717396394","heartbeat_front_addr":"172.21.15.33:6805/2717396394","state":["exists","up"]},{"osd":4,"uuid":"7dc53f55-e2cd-476d-9a81-1e91cdfcae81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6804","nonce":548012972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6808","nonce":548012972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6815","nonce":548012972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":548012972},{"type":"v1","addr":"172.21.15.194:6812","nonce":548012972}]},"public_addr":"172.21.15.194:6804/548012972","cluster_addr":"172.21.15.194:6808/548012972","heartbeat_back_addr":"172.21.15.194:6815/548012972","heartbeat_front_addr":"172.21.15.194:6812/548012972","state":["exists","up"]},{"osd":5,"uuid":"d1e62fa6-2e50-4354-a6d6-089469a34ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6817","nonce":3192443429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6819","nonce":3192443429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6823","nonce":3192443429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3192443429},{"type":"v1","addr":"172.21.15.194:6821","nonce":3192443429}]},"public_addr":"172.21.15.194:6817/3192443429","cluster_addr":"172.21.15.194:6819/3192443429","heartbeat_back_addr":"172.21.15.194:6823/3192443429","heartbeat_front_addr":"172.21.15.194:6821/3192443429","state":["exists","up"]},{"osd":6,"uuid":"e965ff9e-93f4-41ae-be0f-040bf76ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6801","nonce":1743959830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6803","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6805","nonce":1743959830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6811","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6813","nonce":1743959830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6807","nonce":1743959830},{"type":"v1","addr":"172.21.15.194:6809","nonce":1743959830}]},"public_addr":"172.21.15.194:6801/1743959830","cluster_addr":"172.21.15.194:6805/1743959830","heartbeat_back_addr":"172.21.15.194:6813/1743959830","heartbeat_front_addr":"172.21.15.194:6809/1743959830","state":["exists","up"]},{"osd":7,"uuid":"d9c5796e-d4cd-44d4-80ea-3df37f708427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6825","nonce":2849955959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6827","nonce":2849955959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6831","nonce":2849955959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":2849955959},{"type":"v1","addr":"172.21.15.194:6829","nonce":2849955959}]},"public_addr":"172.21.15.194:6825/2849955959","cluster_addr":"172.21.15.194:6827/2849955959","heartbeat_back_addr":"172.21.15.194:6831/2849955959","heartbeat_front_addr":"172.21.15.194:6829/2849955959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.357224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:05.806899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.311947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.339070+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.246563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.337588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.045343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T01:06:06.382385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/4184856115":"2024-03-28T02:08:18.832612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T01:09:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7c2c06bbd0 msgr2=0x7f7c2c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7c2c06bbd0 0x7f7c2c06e090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40008040 comp rx=0 tx=0).stop 2024-03-28T01:09:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500bb0c0 msgr2=0x7f7c501dd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500bb0c0 0x7f7c501dd7b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800c710 tx=0x7f7c4800cbd0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 shutdown_connections 2024-03-28T01:09:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f7c2c06bbd0 0x7f7c2c06e090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c500bb0c0 0x7f7c501dd7b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c500b9a10 0x7f7c500bd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 --2- 172.21.15.33:0/687594129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c500b3070 0x7f7c500bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 >> 172.21.15.33:0/687594129 conn(0x7f7c50072940 msgr2=0x7f7c50072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 shutdown_connections 2024-03-28T01:09:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:00.902+0000 7f7c55c00640 1 -- 172.21.15.33:0/687594129 wait complete. 2024-03-28T01:09:00.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T01:09:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.138+0000 7f48d0a00640 1 Processor -- start 2024-03-28T01:09:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.139+0000 7f48d0a00640 1 -- start start 2024-03-28T01:09:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48d0a00640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48cc0727f0 0x7f48cc141220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48d0a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc0a0cb0 0x7f48cc09fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48d0a00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc072bd0 con 0x7f48cc141760 2024-03-28T01:09:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48d0a00640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0b0d80 con 0x7f48cc0727f0 2024-03-28T01:09:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48d0a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0b1060 con 0x7f48cc0a0cb0 2024-03-28T01:09:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48ca800640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f48cc141760 0x7f48cc0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37366/0 (socket says 172.21.15.33:37366) 2024-03-28T01:09:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48ca800640 1 -- 172.21.15.33:0/1557434826 learned_addr learned my addr 172.21.15.33:0/1557434826 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:09:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.140+0000 7f48c9e00640 1 --2- 172.21.15.33:0/1557434826 >> v2:172.21.15.33:3300/0 conn(0x7f48cc0727f0 0x7f48cc141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.141+0000 7f48c1400640 1 --2- 172.21.15.33:0/1557434826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc0a0cb0 0x7f48cc09fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.141+0000 7f48c9e00640 1 -- 172.21.15.33:0/1557434826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc0a0cb0 msgr2=0x7f48cc09fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:01.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.141+0000 7f48c9e00640 1 --2- 172.21.15.33:0/1557434826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc0a0cb0 0x7f48cc09fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.141+0000 7f48c9400640 1 -- 172.21.15.33:0/1557434826 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071113618 0 0) 0x7f48cc072bd0 con 0x7f48cc141760 2024-03-28T01:09:01.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.141+0000 7f48c9e00640 1 -- 172.21.15.33:0/1557434826 >> v1:172.21.15.33:6789/0 conn(0x7f48cc141760 legacy=0x7f48cc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.141+0000 7f48c9e00640 1 -- 172.21.15.33:0/1557434826 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0b1340 con 0x7f48cc0727f0 2024-03-28T01:09:01.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.141+0000 7f48c1400640 1 --2- 172.21.15.33:0/1557434826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc0a0cb0 0x7f48cc09fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:09:01.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.142+0000 7f48c9e00640 1 --2- 172.21.15.33:0/1557434826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc141220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009d90 tx=0x7f48bc02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bb0dff583f31fa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:01.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.142+0000 7f48c9400640 1 -- 172.21.15.33:0/1557434826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc039070 con 0x7f48cc0727f0 2024-03-28T01:09:01.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.142+0000 7f48c9400640 1 -- 172.21.15.33:0/1557434826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f48bc02cb30 con 0x7f48cc0727f0 2024-03-28T01:09:01.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48c9400640 1 -- 172.21.15.33:0/1557434826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc034440 con 0x7f48cc0727f0 2024-03-28T01:09:01.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 -- 172.21.15.33:0/1557434826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 msgr2=0x7f48cc141220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 --2- 172.21.15.33:0/1557434826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc141220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009d90 tx=0x7f48bc02bd80 comp rx=0 tx=0).stop 2024-03-28T01:09:01.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 -- 172.21.15.33:0/1557434826 shutdown_connections 2024-03-28T01:09:01.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 --2- 172.21.15.33:0/1557434826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc0a0cb0 0x7f48cc09fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 --2- 172.21.15.33:0/1557434826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc141220 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 -- 172.21.15.33:0/1557434826 >> 172.21.15.33:0/1557434826 conn(0x7f48cc06b030 msgr2=0x7f48cc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:01.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 -- 172.21.15.33:0/1557434826 shutdown_connections 2024-03-28T01:09:01.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.143+0000 7f48d0a00640 1 -- 172.21.15.33:0/1557434826 wait complete. 2024-03-28T01:09:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.145+0000 7f48d0a00640 1 Processor -- start 2024-03-28T01:09:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.145+0000 7f48d0a00640 1 -- start start 2024-03-28T01:09:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48d0a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48d0a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48cc0a0cb0 0x7f48cc13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48d0a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc141760 0x7f48cc13d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48d0a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0b0b20 con 0x7f48cc0727f0 2024-03-28T01:09:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48d0a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0b09a0 con 0x7f48cc0a0cb0 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48d0a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0a0290 con 0x7f48cc141760 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48c9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48c9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc143d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36254/0 (socket says 172.21.15.33:36254) 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48c9e00640 1 -- 172.21.15.33:0/2594714856 learned_addr learned my addr 172.21.15.33:0/2594714856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.146+0000 7f48ca800640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc141760 0x7f48cc13d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c1400640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48cc0a0cb0 0x7f48cc13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c9e00640 1 -- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc141760 msgr2=0x7f48cc13d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c9e00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc141760 0x7f48cc13d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c9e00640 1 -- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48cc0a0cb0 msgr2=0x7f48cc13cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c9e00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48cc0a0cb0 0x7f48cc13cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c9e00640 1 -- 172.21.15.33:0/2594714856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc142b70 con 0x7f48cc0727f0 2024-03-28T01:09:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48ca800640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc141760 0x7f48cc13d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T01:09:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c9e00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc143d70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48bc037040 tx=0x7f48bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c2a00640 1 -- 172.21.15.33:0/2594714856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc044020 con 0x7f48cc0727f0 2024-03-28T01:09:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c2a00640 1 -- 172.21.15.33:0/2594714856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f48bc03a040 con 0x7f48cc0727f0 2024-03-28T01:09:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc142e50 con 0x7f48cc0727f0 2024-03-28T01:09:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.147+0000 7f48c2a00640 1 -- 172.21.15.33:0/2594714856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc039050 con 0x7f48cc0727f0 2024-03-28T01:09:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.148+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc141c80 con 0x7f48cc0727f0 2024-03-28T01:09:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.149+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc141f10 con 0x7f48cc0727f0 2024-03-28T01:09:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.151+0000 7f48c2a00640 1 -- 172.21.15.33:0/2594714856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f48bc003bd0 con 0x7f48cc0727f0 2024-03-28T01:09:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.152+0000 7f48c2a00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f48a806bbd0 0x7f48a806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.152+0000 7f48c2a00640 1 -- 172.21.15.33:0/2594714856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f48bc0ad2d0 con 0x7f48cc0727f0 2024-03-28T01:09:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.152+0000 7f48c1400640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f48a806bbd0 0x7f48a806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.153+0000 7f48c1400640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f48a806bbd0 0x7f48a806e090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.156+0000 7f48c2a00640 1 -- 172.21.15.33:0/2594714856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f48bc04e060 con 0x7f48cc0727f0 2024-03-28T01:09:01.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.398+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f48cc143180 con 0x7f48cc0727f0 2024-03-28T01:09:01.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.402+0000 7f48c2a00640 1 -- 172.21.15.33:0/2594714856 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f48bc07bd70 con 0x7f48cc0727f0 2024-03-28T01:09:01.406 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T01:09:01.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.406+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f48a806bbd0 msgr2=0x7f48a806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.406+0000 7f48d0a00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f48a806bbd0 0x7f48a806e090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).stop 2024-03-28T01:09:01.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.406+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 msgr2=0x7f48cc143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.406+0000 7f48d0a00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc143d70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48bc037040 tx=0x7f48bc004a40 comp rx=0 tx=0).stop 2024-03-28T01:09:01.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 shutdown_connections 2024-03-28T01:09:01.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f48a806bbd0 0x7f48a806e090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48cc141760 0x7f48cc13d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48cc0a0cb0 0x7f48cc13cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 --2- 172.21.15.33:0/2594714856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc0727f0 0x7f48cc143d70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 >> 172.21.15.33:0/2594714856 conn(0x7f48cc06b030 msgr2=0x7f48cc13a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 shutdown_connections 2024-03-28T01:09:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.407+0000 7f48d0a00640 1 -- 172.21.15.33:0/2594714856 wait complete. 2024-03-28T01:09:01.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T01:09:01.494 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2024-03-28T01:09:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.621+0000 7f19f9a00640 1 Processor -- start 2024-03-28T01:09:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.621+0000 7f19f9a00640 1 -- start start 2024-03-28T01:09:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.622+0000 7f19f9a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.622+0000 7f19f9a00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f19f41442e0 0x7f19f4144770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.623+0000 7f19f9a00640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f4144cb0 con 0x7f19f406c2b0 2024-03-28T01:09:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.623+0000 7f19f9a00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40b4590 con 0x7f19f41442e0 2024-03-28T01:09:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.623+0000 7f19f9a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f406c690 con 0x7f19f40b1540 2024-03-28T01:09:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.623+0000 7f19f9000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f19f41442e0 0x7f19f4144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.623+0000 7f19f3400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.623+0000 7f19f9000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f19f41442e0 0x7f19f4144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.33:60412/0 (socket says 172.21.15.33:60412) 2024-03-28T01:09:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.623+0000 7f19f9000640 1 -- 172.21.15.33:0/2367345617 learned_addr learned my addr 172.21.15.33:0/2367345617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.624+0000 7f19f3400640 1 -- 172.21.15.33:0/2367345617 >> v2:172.21.15.194:3300/0 conn(0x7f19f41442e0 msgr2=0x7f19f4144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.624+0000 7f19f3400640 1 --2- 172.21.15.33:0/2367345617 >> v2:172.21.15.194:3300/0 conn(0x7f19f41442e0 0x7f19f4144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.624+0000 7f19f3400640 1 -- 172.21.15.33:0/2367345617 >> v1:172.21.15.194:6789/0 conn(0x7f19f406c2b0 legacy=0x7f19f413ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.624+0000 7f19f3400640 1 -- 172.21.15.33:0/2367345617 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40b1b90 con 0x7f19f40b1540 2024-03-28T01:09:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.625+0000 7f19f3400640 1 --2- 172.21.15.33:0/2367345617 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40b47d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba99f07cb69247bb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:01.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.626+0000 7f19f2a00640 1 -- 172.21.15.33:0/2367345617 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0039070 con 0x7f19f40b1540 2024-03-28T01:09:01.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.626+0000 7f19f2a00640 1 -- 172.21.15.33:0/2367345617 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f19e00316f0 con 0x7f19f40b1540 2024-03-28T01:09:01.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.626+0000 7f19f2a00640 1 -- 172.21.15.33:0/2367345617 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0031ae0 con 0x7f19f40b1540 2024-03-28T01:09:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.626+0000 7f19f9a00640 1 -- 172.21.15.33:0/2367345617 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 msgr2=0x7f19f40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.626+0000 7f19f9a00640 1 --2- 172.21.15.33:0/2367345617 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40b47d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002b790 comp rx=0 tx=0).stop 2024-03-28T01:09:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.627+0000 7f19f9a00640 1 -- 172.21.15.33:0/2367345617 shutdown_connections 2024-03-28T01:09:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.627+0000 7f19f9a00640 1 --2- 172.21.15.33:0/2367345617 >> v2:172.21.15.194:3300/0 conn(0x7f19f41442e0 0x7f19f4144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.627+0000 7f19f9a00640 1 --2- 172.21.15.33:0/2367345617 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40b47d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.627+0000 7f19f9a00640 1 -- 172.21.15.33:0/2367345617 >> 172.21.15.33:0/2367345617 conn(0x7f19f40b90b0 msgr2=0x7f19f40b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.627+0000 7f19f9a00640 1 -- 172.21.15.33:0/2367345617 shutdown_connections 2024-03-28T01:09:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.627+0000 7f19f9a00640 1 -- 172.21.15.33:0/2367345617 wait complete. 2024-03-28T01:09:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.629+0000 7f19f9a00640 1 Processor -- start 2024-03-28T01:09:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.629+0000 7f19f9a00640 1 -- start start 2024-03-28T01:09:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19f406c2b0 0x7f19f41de290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40ab030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f41442e0 0x7f19f413eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f413e680 con 0x7f19f41442e0 2024-03-28T01:09:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f413e500 con 0x7f19f406c2b0 2024-03-28T01:09:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f414e7c0 con 0x7f19f40b1540 2024-03-28T01:09:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f41442e0 0x7f19f413eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f41442e0 0x7f19f413eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36276/0 (socket says 172.21.15.33:36276) 2024-03-28T01:09:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f9000640 1 -- 172.21.15.33:0/218635781 learned_addr learned my addr 172.21.15.33:0/218635781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f3e00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19f406c2b0 0x7f19f41de290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.630+0000 7f19f3400640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f9000640 1 -- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 msgr2=0x7f19f40ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f9000640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f9000640 1 -- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19f406c2b0 msgr2=0x7f19f41de290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:01.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f9000640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19f406c2b0 0x7f19f41de290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:01.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f9000640 1 -- 172.21.15.33:0/218635781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40abad0 con 0x7f19f41442e0 2024-03-28T01:09:01.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f3400640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:09:01.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f3e00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19f406c2b0 0x7f19f41de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:09:01.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19f9000640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f41442e0 0x7f19f413eb70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e870 tx=0x7f19ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19ebe00640 1 -- 172.21.15.33:0/218635781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec017070 con 0x7f19f41442e0 2024-03-28T01:09:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19ebe00640 1 -- 172.21.15.33:0/218635781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f19ec00c940 con 0x7f19f41442e0 2024-03-28T01:09:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.631+0000 7f19ebe00640 1 -- 172.21.15.33:0/218635781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00cd50 con 0x7f19f41442e0 2024-03-28T01:09:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.632+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f41deb60 con 0x7f19f41442e0 2024-03-28T01:09:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.632+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f41de7d0 con 0x7f19f41442e0 2024-03-28T01:09:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.633+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f404f520 con 0x7f19f41442e0 2024-03-28T01:09:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.634+0000 7f19ebe00640 1 -- 172.21.15.33:0/218635781 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f19ec01a070 con 0x7f19f41442e0 2024-03-28T01:09:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.635+0000 7f19ebe00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f19d406bc00 0x7f19d406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.635+0000 7f19ebe00640 1 -- 172.21.15.33:0/218635781 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6012+0+0 (secure 0 0 0) 0x7f19ec08aee0 con 0x7f19f41442e0 2024-03-28T01:09:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.635+0000 7f19f3e00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f19d406bc00 0x7f19d406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:01.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.636+0000 7f19f3e00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f19d406bc00 0x7f19d406e0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19e4001d60 tx=0x7f19e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:01.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.639+0000 7f19ebe00640 1 -- 172.21.15.33:0/218635781 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f19f404f520 con 0x7f19f41442e0 2024-03-28T01:09:01.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:01.884+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f19f4140820 con 0x7f19f41442e0 2024-03-28T01:09:02.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.406+0000 7f19ebe00640 1 -- 172.21.15.33:0/218635781 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f19ec059930 con 0x7f19f41442e0 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f19d406bc00 msgr2=0x7f19d406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f19d406bc00 0x7f19d406e0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19e4001d60 tx=0x7f19e4009040 comp rx=0 tx=0).stop 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f41442e0 msgr2=0x7f19f413eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f41442e0 0x7f19f413eb70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e870 tx=0x7f19ec00ed30 comp rx=0 tx=0).stop 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 shutdown_connections 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f19d406bc00 0x7f19d406e0c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f41442e0 0x7f19f413eb70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19f40b1540 0x7f19f40ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 --2- 172.21.15.33:0/218635781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19f406c2b0 0x7f19f41de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 >> 172.21.15.33:0/218635781 conn(0x7f19f40b90b0 msgr2=0x7f19f40b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:02.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 shutdown_connections 2024-03-28T01:09:02.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.409+0000 7f19f9a00640 1 -- 172.21.15.33:0/218635781 wait complete. 2024-03-28T01:09:02.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T01:09:02.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.647+0000 7f189a000640 1 Processor -- start 2024-03-28T01:09:02.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.648+0000 7f189a000640 1 -- start start 2024-03-28T01:09:02.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.649+0000 7f189a000640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7f189406f1d0 0x7f189413ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.649+0000 7f189a000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18940b2620 con 0x7f189413f2d0 2024-03-28T01:09:02.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.649+0000 7f189a000640 1 -- --> v1:172.21.15.194:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189406cf70 con 0x7f18940b2240 2024-03-28T01:09:02.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.649+0000 7f189a000640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189406be00 con 0x7f189406f1d0 2024-03-28T01:09:02.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.649+0000 7f1899600640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f189413f2d0 0x7f189414f7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46584/0 (socket says 172.21.15.33:46584) 2024-03-28T01:09:02.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.649+0000 7f1899600640 1 -- 172.21.15.33:0/1911282517 learned_addr learned my addr 172.21.15.33:0/1911282517 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:09:02.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893e00640 1 --2- 172.21.15.33:0/1911282517 >> v2:172.21.15.194:3300/0 conn(0x7f189406f1d0 0x7f189413ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866983959 0 0) 0x7f18940b2620 con 0x7f189413f2d0 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1878003520 con 0x7f189413f2d0 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1920757890 0 0) 0x7f1878003520 con 0x7f189413f2d0 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 >> v2:172.21.15.194:3300/0 conn(0x7f189406f1d0 msgr2=0x7f189413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 --2- 172.21.15.33:0/1911282517 >> v2:172.21.15.194:3300/0 conn(0x7f189406f1d0 0x7f189413ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 >> v1:172.21.15.194:6789/0 conn(0x7f18940b2240 legacy=0x7f1894144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189406b600 con 0x7f189413f2d0 2024-03-28T01:09:02.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7f188c001670 con 0x7f189413f2d0 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 >> v1:172.21.15.33:6789/0 conn(0x7f189413f2d0 legacy=0x7f189414f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18780039b0 0x7f1878003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1878004800 0x7f1878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780051c0 0x7f187800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189406be00 con 0x7f18780039b0 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187800c160 con 0x7f18780051c0 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.650+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187800c360 con 0x7f1878004800 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1898c00640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18780039b0 0x7f1878003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1899600640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780051c0 0x7f187800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1893e00640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1878004800 0x7f1878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1898c00640 1 -- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1878004800 msgr2=0x7f1878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1898c00640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1878004800 0x7f1878004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1898c00640 1 -- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780051c0 msgr2=0x7f187800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1898c00640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780051c0 0x7f187800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1899600640 1 -- 172.21.15.33:0/1911282517 reap_dead start 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.651+0000 7f1898c00640 1 -- 172.21.15.33:0/1911282517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878003520 con 0x7f18780039b0 2024-03-28T01:09:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.652+0000 7f1898c00640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18780039b0 0x7f1878003dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f187c001750 tx=0x7f1878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8cca616042c5ac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.652+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c001370 con 0x7f18780039b0 2024-03-28T01:09:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f187c0052d0 con 0x7f18780039b0 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f1893400640 1 -- 172.21.15.33:0/1911282517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c003850 con 0x7f18780039b0 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f189a000640 1 -- 172.21.15.33:0/1911282517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18780039b0 msgr2=0x7f1878003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f189a000640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18780039b0 0x7f1878003dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f187c001750 tx=0x7f1878005830 comp rx=0 tx=0).stop 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f189a000640 1 -- 172.21.15.33:0/1911282517 shutdown_connections 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f189a000640 1 --2- 172.21.15.33:0/1911282517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18780039b0 0x7f1878003dd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f189a000640 1 -- 172.21.15.33:0/1911282517 >> 172.21.15.33:0/1911282517 conn(0x7f18940b55a0 msgr2=0x7f18940b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f189a000640 1 -- 172.21.15.33:0/1911282517 shutdown_connections 2024-03-28T01:09:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.653+0000 7f189a000640 1 -- 172.21.15.33:0/1911282517 wait complete. 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.655+0000 7f189a000640 1 Processor -- start 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.655+0000 7f189a000640 1 -- start start 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f189a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780039b0 0x7f1894148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f189a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1894143070 0x7f18941dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f189a000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941dd1a0 0x7f1894141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f189a000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189406cd30 con 0x7f18941dd1a0 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f189a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189406cbb0 con 0x7f18780039b0 2024-03-28T01:09:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f189a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189406b030 con 0x7f1894143070 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f1899600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941dd1a0 0x7f1894141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f1899600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941dd1a0 0x7f1894141af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34412/0 (socket says 172.21.15.33:34412) 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f1899600640 1 -- 172.21.15.33:0/1137923965 learned_addr learned my addr 172.21.15.33:0/1137923965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f1898c00640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780039b0 0x7f1894148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f1893e00640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1894143070 0x7f18941dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.656+0000 7f1899600640 1 -- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1894143070 msgr2=0x7f18941dcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1899600640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1894143070 0x7f18941dcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1899600640 1 -- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780039b0 msgr2=0x7f1894148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1899600640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780039b0 0x7f1894148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1899600640 1 -- 172.21.15.33:0/1137923965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894145d40 con 0x7f18941dd1a0 2024-03-28T01:09:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1899600640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941dd1a0 0x7f1894141af0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f189406b570 tx=0x7f188c00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1890c00640 1 -- 172.21.15.33:0/1137923965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c019070 con 0x7f18941dd1a0 2024-03-28T01:09:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1890c00640 1 -- 172.21.15.33:0/1137923965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f188c010510 con 0x7f18941dd1a0 2024-03-28T01:09:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f1890c00640 1 -- 172.21.15.33:0/1137923965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c010920 con 0x7f18941dd1a0 2024-03-28T01:09:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1894143e40 con 0x7f18941dd1a0 2024-03-28T01:09:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.657+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1894142d30 con 0x7f18941dd1a0 2024-03-28T01:09:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.659+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1894075310 con 0x7f18941dd1a0 2024-03-28T01:09:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.659+0000 7f1890c00640 1 -- 172.21.15.33:0/1137923965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f188c010ac0 con 0x7f18941dd1a0 2024-03-28T01:09:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.660+0000 7f1890c00640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f187006bc00 0x7f187006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.660+0000 7f1890c00640 1 -- 172.21.15.33:0/1137923965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6012+0+0 (secure 0 0 0) 0x7f188c089f60 con 0x7f18941dd1a0 2024-03-28T01:09:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.661+0000 7f1898c00640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f187006bc00 0x7f187006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.661+0000 7f1898c00640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f187006bc00 0x7f187006e0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f187c033000 tx=0x7f187c009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.665+0000 7f1890c00640 1 -- 172.21.15.33:0/1137923965 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f188c058230 con 0x7f18941dd1a0 2024-03-28T01:09:02.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:02.871+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18940b42d0 con 0x7f18941dd1a0 2024-03-28T01:09:03.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.519+0000 7f1890c00640 1 -- 172.21.15.33:0/1137923965 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f188c05c2d0 con 0x7f18941dd1a0 2024-03-28T01:09:03.523 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-03-28T01:09:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.525+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f187006bc00 msgr2=0x7f187006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.525+0000 7f189a000640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f187006bc00 0x7f187006e0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f187c033000 tx=0x7f187c009030 comp rx=0 tx=0).stop 2024-03-28T01:09:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.525+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941dd1a0 msgr2=0x7f1894141af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.525+0000 7f189a000640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941dd1a0 0x7f1894141af0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f189406b570 tx=0x7f188c00e960 comp rx=0 tx=0).stop 2024-03-28T01:09:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 shutdown_connections 2024-03-28T01:09:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7f187006bc00 0x7f187006e0c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941dd1a0 0x7f1894141af0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1894143070 0x7f18941dcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 --2- 172.21.15.33:0/1137923965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18780039b0 0x7f1894148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 >> 172.21.15.33:0/1137923965 conn(0x7f18940b55a0 msgr2=0x7f18940b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 shutdown_connections 2024-03-28T01:09:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.526+0000 7f189a000640 1 -- 172.21.15.33:0/1137923965 wait complete. 2024-03-28T01:09:03.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T01:09:03.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.761+0000 7ff82ee00640 1 Processor -- start 2024-03-28T01:09:03.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.761+0000 7ff82ee00640 1 -- start start 2024-03-28T01:09:03.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.762+0000 7ff82ee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 0x7ff8280b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.762+0000 7ff82ee00640 1 --2- >> v2:172.21.15.194:3300/0 conn(0x7ff828144040 0x7ff828144470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.762+0000 7ff82ee00640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8280b4920 con 0x7ff8280b07c0 2024-03-28T01:09:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff82ee00640 1 -- --> v2:172.21.15.194:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82806f470 con 0x7ff828144040 2024-03-28T01:09:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff82ee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281449b0 con 0x7ff8280b4540 2024-03-28T01:09:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff82d000640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff8280b07c0 0x7ff828140fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46596/0 (socket says 172.21.15.33:46596) 2024-03-28T01:09:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff82d000640 1 -- 172.21.15.33:0/235942774 learned_addr learned my addr 172.21.15.33:0/235942774 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-03-28T01:09:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff82e400640 1 --2- 172.21.15.33:0/235942774 >> v2:172.21.15.194:3300/0 conn(0x7ff828144040 0x7ff828144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff82da00640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 0x7ff8280b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085482199 0 0) 0x7ff8280b4920 con 0x7ff8280b07c0 2024-03-28T01:09:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.763+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff810003520 con 0x7ff8280b07c0 2024-03-28T01:09:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387828396 0 0) 0x7ff810003520 con 0x7ff8280b07c0 2024-03-28T01:09:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 msgr2=0x7ff8280b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 0x7ff8280b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 >> v2:172.21.15.194:3300/0 conn(0x7ff828144040 msgr2=0x7ff828144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 --2- 172.21.15.33:0/235942774 >> v2:172.21.15.194:3300/0 conn(0x7ff828144040 0x7ff828144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280b19d0 con 0x7ff8280b07c0 2024-03-28T01:09:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1648271721 0 0) 0x7ff814001670 con 0x7ff8280b07c0 2024-03-28T01:09:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 >> v1:172.21.15.33:6789/0 conn(0x7ff8280b07c0 legacy=0x7ff828140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 0x7ff8100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810003f80 0x7ff810004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810004d40 0x7ff8100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82806f470 con 0x7ff810004d40 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281449b0 con 0x7ff810003f80 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81000dd00 con 0x7ff8280b4540 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff82e400640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810004d40 0x7ff8100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.764+0000 7ff82d000640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810003f80 0x7ff810004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.765+0000 7ff82da00640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 0x7ff8100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.765+0000 7ff82e400640 1 -- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 msgr2=0x7ff8100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.765+0000 7ff82e400640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280b4540 0x7ff8100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.765+0000 7ff82e400640 1 -- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810003f80 msgr2=0x7ff810004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T01:09:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.765+0000 7ff82e400640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810003f80 0x7ff810004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.765+0000 7ff82e400640 1 -- 172.21.15.33:0/235942774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810003520 con 0x7ff810004d40 2024-03-28T01:09:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.765+0000 7ff82d000640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810003f80 0x7ff810004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.766+0000 7ff82e400640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810004d40 0x7ff8100051c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fc0 tx=0x7ff810005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dbdaeb3a948939 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.766+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820039070 con 0x7ff810004d40 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.766+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff82002cc60 con 0x7ff810004d40 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.766+0000 7ff827e00640 1 -- 172.21.15.33:0/235942774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820009c10 con 0x7ff810004d40 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.767+0000 7ff82ee00640 1 -- 172.21.15.33:0/235942774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810004d40 msgr2=0x7ff8100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.767+0000 7ff82ee00640 1 --2- 172.21.15.33:0/235942774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810004d40 0x7ff8100051c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fc0 tx=0x7ff810005830 comp rx=0 tx=0).stop 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.767+0000 7ff82e400640 1 -- 172.21.15.33:0/235942774 reap_dead start 2024-03-28T01:09:03.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.767+0000 7ff82ee00640 1 -- 172.21.15.33:0/235942774 shutdown_connections 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.767+0000 7ff82ee00640 1 -- 172.21.15.33:0/235942774 >> 172.21.15.33:0/235942774 conn(0x7ff8280b55a0 msgr2=0x7ff8280b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.767+0000 7ff82ee00640 1 -- 172.21.15.33:0/235942774 shutdown_connections 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.767+0000 7ff82ee00640 1 -- 172.21.15.33:0/235942774 wait complete. 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.769+0000 7ff82ee00640 1 Processor -- start 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.769+0000 7ff82ee00640 1 -- start start 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.769+0000 7ff82ee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810004d40 0x7ff8280b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82ee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280705d0 0x7ff8281d8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82ee00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828070a50 0x7ff8281d88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82ee00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281416e0 con 0x7ff828070a50 2024-03-28T01:09:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82ee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280b1f90 con 0x7ff810004d40 2024-03-28T01:09:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82ee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff828141860 con 0x7ff8280705d0 2024-03-28T01:09:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82e400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828070a50 0x7ff8281d88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82e400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828070a50 0x7ff8281d88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34424/0 (socket says 172.21.15.33:34424) 2024-03-28T01:09:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82e400640 1 -- 172.21.15.33:0/2698633753 learned_addr learned my addr 172.21.15.33:0/2698633753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-03-28T01:09:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82da00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810004d40 0x7ff8280b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82d000640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280705d0 0x7ff8281d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82e400640 1 -- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280705d0 msgr2=0x7ff8281d8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82e400640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280705d0 0x7ff8281d8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.770+0000 7ff82e400640 1 -- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810004d40 msgr2=0x7ff8280b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff82e400640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810004d40 0x7ff8280b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff82e400640 1 -- 172.21.15.33:0/2698633753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828139640 con 0x7ff828070a50 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff82d000640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280705d0 0x7ff8281d8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff82e400640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828070a50 0x7ff8281d88b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff820002230 tx=0x7ff820009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff825600640 1 -- 172.21.15.33:0/2698633753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820043030 con 0x7ff828070a50 2024-03-28T01:09:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff825600640 1 -- 172.21.15.33:0/2698633753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff82002ca60 con 0x7ff828070a50 2024-03-28T01:09:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff825600640 1 -- 172.21.15.33:0/2698633753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820039040 con 0x7ff828070a50 2024-03-28T01:09:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff828070f30 con 0x7ff828070a50 2024-03-28T01:09:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.771+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82806fce0 con 0x7ff828070a50 2024-03-28T01:09:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.773+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff828075310 con 0x7ff828070a50 2024-03-28T01:09:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.773+0000 7ff825600640 1 -- 172.21.15.33:0/2698633753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff820046070 con 0x7ff828070a50 2024-03-28T01:09:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.775+0000 7ff825600640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff80806bc00 0x7ff80806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T01:09:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.775+0000 7ff825600640 1 -- 172.21.15.33:0/2698633753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5574+0+0 (secure 0 0 0) 0x7ff8200ac870 con 0x7ff828070a50 2024-03-28T01:09:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.775+0000 7ff82da00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff80806bc00 0x7ff80806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T01:09:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.776+0000 7ff82da00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff80806bc00 0x7ff80806e0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009a40 tx=0x7ff81c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T01:09:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.779+0000 7ff825600640 1 -- 172.21.15.33:0/2698633753 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff82004e020 con 0x7ff828070a50 2024-03-28T01:09:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:03.989+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff8281436e0 con 0x7ff828070a50 2024-03-28T01:09:04.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.529+0000 7ff825600640 1 -- 172.21.15.33:0/2698633753 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff82002cc30 con 0x7ff828070a50 2024-03-28T01:09:04.533 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-03-28T01:09:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff80806bc00 msgr2=0x7ff80806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff80806bc00 0x7ff80806e0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009a40 tx=0x7ff81c008040 comp rx=0 tx=0).stop 2024-03-28T01:09:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828070a50 msgr2=0x7ff8281d88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T01:09:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828070a50 0x7ff8281d88b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff820002230 tx=0x7ff820009750 comp rx=0 tx=0).stop 2024-03-28T01:09:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 shutdown_connections 2024-03-28T01:09:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:6832/199212674,v1:172.21.15.33:6833/199212674] conn(0x7ff80806bc00 0x7ff80806e0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828070a50 0x7ff8281d88b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8280705d0 0x7ff8281d8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 --2- 172.21.15.33:0/2698633753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff810004d40 0x7ff8280b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T01:09:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.533+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 >> 172.21.15.33:0/2698633753 conn(0x7ff8280b55a0 msgr2=0x7ff82813d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T01:09:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.535+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 shutdown_connections 2024-03-28T01:09:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-03-28T01:09:04.536+0000 7ff82ee00640 1 -- 172.21.15.33:0/2698633753 wait complete. 2024-03-28T01:09:04.570 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-28T01:09:04.571 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T01:09:04.571 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.590 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T01:09:04.591 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T01:09:04.591 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.612 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T01:09:04.612 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-28T01:09:04.613 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-28T01:09:04.613 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.676 INFO:tasks.ceph.osd.0:Stopped 2024-03-28T01:09:04.676 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-28T01:09:04.676 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.739 INFO:tasks.ceph.osd.1:Stopped 2024-03-28T01:09:04.740 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-28T01:09:04.740 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.791 INFO:tasks.ceph.osd.2:Stopped 2024-03-28T01:09:04.791 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-28T01:09:04.792 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.841 INFO:tasks.ceph.osd.3:Stopped 2024-03-28T01:09:04.841 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-28T01:09:04.841 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.921 INFO:tasks.ceph.osd.4:Stopped 2024-03-28T01:09:04.921 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-28T01:09:04.921 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:04.999 INFO:tasks.ceph.osd.5:Stopped 2024-03-28T01:09:04.999 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-28T01:09:05.000 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:05.062 INFO:tasks.ceph.osd.6:Stopped 2024-03-28T01:09:05.063 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-28T01:09:05.063 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:05.129 INFO:tasks.ceph.osd.7:Stopped 2024-03-28T01:09:05.129 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-28T01:09:05.129 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-28T01:09:05.129 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:05.213 INFO:tasks.ceph.mgr.y:Stopped 2024-03-28T01:09:05.214 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-28T01:09:05.214 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:05.297 INFO:tasks.ceph.mgr.x:Stopped 2024-03-28T01:09:05.298 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-28T01:09:05.298 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-28T01:09:05.298 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:05.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-03-28T01:09:05.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-03-28T01:09:05.347 INFO:tasks.ceph.mon.a:Stopped 2024-03-28T01:09:05.347 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-28T01:09:05.347 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:05.382 INFO:tasks.ceph.mon.b:Stopped 2024-03-28T01:09:05.382 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-28T01:09:05.383 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T01:09:05.448 INFO:tasks.ceph.mon.c:Stopped 2024-03-28T01:09:05.448 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-28T01:09:05.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-28T01:09:05.560 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:09:05.560 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-28T01:09:05.872 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:09:05.873 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-28T01:09:06.032 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:09:06.033 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-28T01:09:06.196 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:09:06.196 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-28T01:09:06.388 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:09:06.389 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-28T01:09:06.639 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:09:06.639 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-28T01:09:06.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:09:06.797 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-28T01:09:06.933 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:09:06.934 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-28T01:09:07.112 INFO:tasks.ceph:Archiving mon data... 2024-03-28T01:09:07.113 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/data/mon.a.tgz 2024-03-28T01:09:07.113 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-03-28T01:09:07.173 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.4IN3TZjCgR 2024-03-28T01:09:07.174 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4IN3TZjCgR 2024-03-28T01:09:07.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.4IN3TZjCgR 2024-03-28T01:09:07.516 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.4IN3TZjCgR is 260KB 2024-03-28T01:09:07.628 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.4IN3TZjCgR 2024-03-28T01:09:07.685 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/data/mon.b.tgz 2024-03-28T01:09:07.686 DEBUG:teuthology.orchestra.run.smithi194:> mktemp 2024-03-28T01:09:07.742 INFO:teuthology.orchestra.run.smithi194.stdout:/tmp/tmp.LMSQnURn50 2024-03-28T01:09:07.743 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.LMSQnURn50 2024-03-28T01:09:07.928 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0666 /tmp/tmp.LMSQnURn50 2024-03-28T01:09:08.077 DEBUG:teuthology.orchestra.remote:smithi194:/tmp/tmp.LMSQnURn50 is 356KB 2024-03-28T01:09:08.188 DEBUG:teuthology.orchestra.run.smithi194:> rm -fr /tmp/tmp.LMSQnURn50 2024-03-28T01:09:08.246 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/data/mon.c.tgz 2024-03-28T01:09:08.246 DEBUG:teuthology.orchestra.run.smithi194:> mktemp 2024-03-28T01:09:08.298 INFO:teuthology.orchestra.run.smithi194.stdout:/tmp/tmp.YmiKCpPJkE 2024-03-28T01:09:08.299 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.YmiKCpPJkE 2024-03-28T01:09:08.471 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0666 /tmp/tmp.YmiKCpPJkE 2024-03-28T01:09:08.618 DEBUG:teuthology.orchestra.remote:smithi194:/tmp/tmp.YmiKCpPJkE is 356KB 2024-03-28T01:09:08.722 DEBUG:teuthology.orchestra.run.smithi194:> rm -fr /tmp/tmp.YmiKCpPJkE 2024-03-28T01:09:08.779 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-28T01:09:08.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-28T01:09:08.787 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-28T01:09:08.938 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-28T01:09:08.997 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-28T01:09:08.997 INFO:tasks.ceph:Archiving crash dumps... 2024-03-28T01:09:08.999 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/remote/smithi033/crash 2024-03-28T01:09:08.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-28T01:09:09.105 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/remote/smithi194/crash 2024-03-28T01:09:09.106 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-28T01:09:09.200 INFO:tasks.ceph:Shutting down logrotate 2024-03-28T01:09:09.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:09:09.208 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T01:09:09.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-28T01:09:09.444 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-28T01:09:09.543 INFO:tasks.ceph:Compressing logs... 2024-03-28T01:09:09.543 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T01:09:09.551 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T01:09:09.636 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-28T01:09:09.637 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37177.log 2024-03-28T01:09:09.638 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37057.log 2024-03-28T01:09:09.639 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.37177.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37177.log.gz 2024-03-28T01:09:09.640 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37257.log 2024-03-28T01:09:09.641 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.37057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37137.log 2024-03-28T01:09:09.641 INFO:teuthology.orchestra.run.smithi194.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37057.log.gz 2024-03-28T01:09:09.642 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.37257.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37257.log.gz 2024-03-28T01:09:09.642 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37017.log 2024-03-28T01:09:09.643 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.37137.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-28T01:09:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43948.log 2024-03-28T01:09:09.644 INFO:teuthology.orchestra.run.smithi194.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37137.log.gz 2024-03-28T01:09:09.644 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.37017.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-28T01:09:09.645 INFO:teuthology.orchestra.run.smithi194.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37017.log.gz 2024-03-28T01:09:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-28T01:09:09.646 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37217.log 2024-03-28T01:09:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44368.log 2024-03-28T01:09:09.647 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43948.log.gz 2024-03-28T01:09:09.647 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-mds.b.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-28T01:09:09.647 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-28T01:09:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44790.log 2024-03-28T01:09:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46373.log 2024-03-28T01:09:09.649 INFO:teuthology.orchestra.run.smithi194.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-28T01:09:09.649 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.37217.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-28T01:09:09.649 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-osd.4.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37217.log.gz 2024-03-28T01:09:09.649 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-28T01:09:09.651 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37097.log 2024-03-28T01:09:09.651 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44790.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44368.log.gz 2024-03-28T01:09:09.651 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 2024-03-28T01:09:09.651 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43635.log 2024-03-28T01:09:09.652 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46373.log.gz 2024-03-28T01:09:09.653 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43690.log 2024-03-28T01:09:09.653 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-03-28T01:09:09.653 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-28T01:09:09.654 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.37097.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-28T01:09:09.654 INFO:teuthology.orchestra.run.smithi194.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37097.log.gz 2024-03-28T01:09:09.655 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-28T01:09:09.655 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42432.log 2024-03-28T01:09:09.656 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43635.log.gz 2024-03-28T01:09:09.656 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43690.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43690.log.gz 2024-03-28T01:09:09.656 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-03-28T01:09:09.657 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36977.log 2024-03-28T01:09:09.657 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-03-28T01:09:09.658 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42432.log.gz 2024-03-28T01:09:09.659 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-mon.b.log: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-28T01:09:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44284.log 2024-03-28T01:09:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43029.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gz 2024-03-28T01:09:09.660 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-03-28T01:09:09.661 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45624.log 2024-03-28T01:09:09.661 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.36977.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36977.log.gz 2024-03-28T01:09:09.661 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-28T01:09:09.661 INFO:teuthology.orchestra.run.smithi194.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-28T01:09:09.662 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44032.log 2024-03-28T01:09:09.662 INFO:teuthology.orchestra.run.smithi033.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44284.log.gz 2024-03-28T01:09:09.663 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-03-28T01:09:09.663 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45624.log.gz 2024-03-28T01:09:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46062.log 2024-03-28T01:09:09.665 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44032.log.gz 2024-03-28T01:09:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46144.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-03-28T01:09:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43329.log 2024-03-28T01:09:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43681.log 2024-03-28T01:09:09.668 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46062.log.gz 2024-03-28T01:09:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43329.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43329.log.gz 2024-03-28T01:09:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43347.log 2024-03-28T01:09:09.670 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42814.log 2024-03-28T01:09:09.671 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43681.log.gz 2024-03-28T01:09:09.673 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43347.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.43347.log.gz 2024-03-28T01:09:09.673 INFO:teuthology.orchestra.run.smithi033.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44116.log 2024-03-28T01:09:09.673 INFO:teuthology.orchestra.run.smithi033.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.44790.log.gz 2024-03-28T01:09:09.674 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2024-03-28T01:09:09.674 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42814.log.gz 2024-03-28T01:09:09.675 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43864.log 2024-03-28T01:09:09.675 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44116.log.gz 2024-03-28T01:09:09.678 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42520.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2024-03-28T01:09:09.678 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45666.log 2024-03-28T01:09:09.678 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42209.log 2024-03-28T01:09:09.678 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43864.log.gz 2024-03-28T01:09:09.680 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43031.log 2024-03-28T01:09:09.680 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45666.log.gz 2024-03-28T01:09:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42209.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42209.log.gz 2024-03-28T01:09:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45246.log 2024-03-28T01:09:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43346.log 2024-03-28T01:09:09.683 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43031.log.gz 2024-03-28T01:09:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45246.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45246.log.gz 2024-03-28T01:09:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-28T01:09:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46228.log 2024-03-28T01:09:09.686 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43346.log.gz 2024-03-28T01:09:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2024-03-28T01:09:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46228.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-28T01:09:09.687 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46228.log.gz 2024-03-28T01:09:09.687 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45918.log 2024-03-28T01:09:09.688 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42021.log 2024-03-28T01:09:09.688 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gz 2024-03-28T01:09:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45918.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45918.log.gz 2024-03-28T01:09:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46186.log 2024-03-28T01:09:09.691 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42021.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42021.log.gz 2024-03-28T01:09:09.692 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43906.log 2024-03-28T01:09:09.692 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2024-03-28T01:09:09.692 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46186.log.gz 2024-03-28T01:09:09.692 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43906.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43906.log.gz 2024-03-28T01:09:09.692 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45456.log 2024-03-28T01:09:09.693 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43038.log 2024-03-28T01:09:09.694 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2024-03-28T01:09:09.695 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43033.log 2024-03-28T01:09:09.695 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45456.log.gz 2024-03-28T01:09:09.696 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42083.log 2024-03-28T01:09:09.696 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43038.log.gz 2024-03-28T01:09:09.697 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45582.log 2024-03-28T01:09:09.697 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42083.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43033.log.gz 2024-03-28T01:09:09.698 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42083.log.gz 2024-03-28T01:09:09.698 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42856.log 2024-03-28T01:09:09.700 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45540.log 2024-03-28T01:09:09.700 INFO:teuthology.orchestra.run.smithi033.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45582.log.gz 2024-03-28T01:09:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45372.log 2024-03-28T01:09:09.701 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42856.log.gz 2024-03-28T01:09:09.702 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2024-03-28T01:09:09.702 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45540.log.gz 2024-03-28T01:09:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45372.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45372.log.gz 2024-03-28T01:09:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42604.log 2024-03-28T01:09:09.705 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-03-28T01:09:09.706 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2024-03-28T01:09:09.706 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45288.log 2024-03-28T01:09:09.706 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42604.log.gz 2024-03-28T01:09:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43354.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30921.log 2024-03-28T01:09:09.707 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-03-28T01:09:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44494.log 2024-03-28T01:09:09.708 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45288.log.gz 2024-03-28T01:09:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.tmp-client.admin.30921.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30921.log.gz 2024-03-28T01:09:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-28T01:09:09.711 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2024-03-28T01:09:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.1.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44494.log.gz 2024-03-28T01:09:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46415.log 2024-03-28T01:09:09.713 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43677.log 2024-03-28T01:09:09.713 INFO:teuthology.orchestra.run.smithi033.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2024-03-28T01:09:09.714 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42772.log 2024-03-28T01:09:09.715 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46415.log.gz 2024-03-28T01:09:09.716 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43677.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43677.log.gz 2024-03-28T01:09:09.716 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43034.log 2024-03-28T01:09:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43035.log 2024-03-28T01:09:09.718 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42772.log.gz 2024-03-28T01:09:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43034.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43034.log.gz 2024-03-28T01:09:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46330.log 2024-03-28T01:09:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45204.log 2024-03-28T01:09:09.721 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43035.log.gz 2024-03-28T01:09:09.721 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46330.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46330.log.gz 2024-03-28T01:09:09.722 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43678.log 2024-03-28T01:09:09.722 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41944.log 2024-03-28T01:09:09.723 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45204.log.gz 2024-03-28T01:09:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43678.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43678.log.gz 2024-03-28T01:09:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42688.log 2024-03-28T01:09:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.41944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45162.log 2024-03-28T01:09:09.726 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41944.log.gz 2024-03-28T01:09:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42688.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42688.log.gz 2024-03-28T01:09:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-28T01:09:09.729 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46002.log 2024-03-28T01:09:09.730 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45162.log.gz 2024-03-28T01:09:09.730 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46270.log 2024-03-28T01:09:09.732 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46002.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46002.log.gz 2024-03-28T01:09:09.732 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36997.log 2024-03-28T01:09:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42562.log 2024-03-28T01:09:09.735 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46270.log.gz 2024-03-28T01:09:09.735 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36997.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36997.log.gz 2024-03-28T01:09:09.735 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37113.log 2024-03-28T01:09:09.736 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-03-28T01:09:09.736 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42562.log.gz 2024-03-28T01:09:09.738 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37113.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37113.log.gz 2024-03-28T01:09:09.738 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-28T01:09:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42646.log 2024-03-28T01:09:09.739 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-03-28T01:09:09.740 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45498.log 2024-03-28T01:09:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42646.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42646.log.gzgzip 2024-03-28T01:09:09.742 INFO:teuthology.orchestra.run.smithi033.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42474.log 2024-03-28T01:09:09.743 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45414.log 2024-03-28T01:09:09.744 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-mon.c.log: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-28T01:09:09.744 INFO:teuthology.orchestra.run.smithi194.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-28T01:09:09.745 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45498.log.gz 2024-03-28T01:09:09.745 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44326.log 2024-03-28T01:09:09.745 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42474.log.gz 2024-03-28T01:09:09.747 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42293.log 2024-03-28T01:09:09.747 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45414.log.gz 2024-03-28T01:09:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44200.log 2024-03-28T01:09:09.749 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44326.log.gz 2024-03-28T01:09:09.749 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-03-28T01:09:09.750 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42293.log.gz 2024-03-28T01:09:09.751 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45834.log 2024-03-28T01:09:09.752 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44200.log.gz 2024-03-28T01:09:09.752 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43689.log 2024-03-28T01:09:09.753 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-03-28T01:09:09.754 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-03-28T01:09:09.754 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45834.log.gz 2024-03-28T01:09:09.756 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-03-28T01:09:09.756 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43689.log.gz 2024-03-28T01:09:09.757 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42381.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-28T01:09:09.757 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gz 2024-03-28T01:09:09.758 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44242.log 2024-03-28T01:09:09.758 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-03-28T01:09:09.759 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-03-28T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43353.log 2024-03-28T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44242.log.gz 2024-03-28T01:09:09.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-03-28T01:09:09.763 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-03-28T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43353.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-28T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43353.log.gz 2024-03-28T01:09:09.766 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43351.log 2024-03-28T01:09:09.766 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-03-28T01:09:09.769 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2024-03-28T01:09:09.769 INFO:teuthology.orchestra.run.smithi033.stderr: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-28T01:09:09.769 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43351.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-28T01:09:09.769 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43351.log.gz 2024-03-28T01:09:09.770 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43503.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2024-03-28T01:09:09.770 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44158.log 2024-03-28T01:09:09.771 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42251.log 2024-03-28T01:09:09.773 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44158.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44158.log.gz 2024-03-28T01:09:09.773 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45960.log 2024-03-28T01:09:09.775 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42251.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42251.log.gz 2024-03-28T01:09:09.776 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.45960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36743.log 2024-03-28T01:09:09.776 INFO:teuthology.orchestra.run.smithi033.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45960.log.gz 2024-03-28T01:09:09.778 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36743.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36743.log.gz 2024-03-28T01:09:09.779 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44452.log 2024-03-28T01:09:09.780 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42730.log 2024-03-28T01:09:09.781 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44452.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44452.log.gz 2024-03-28T01:09:09.783 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42730.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42730.log.gz 2024-03-28T01:09:09.783 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43022.log 2024-03-28T01:09:09.785 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42125.log 2024-03-28T01:09:09.786 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43022.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43022.log.gz 2024-03-28T01:09:09.788 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.42125.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42125.log.gz 2024-03-28T01:09:09.788 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-03-28T01:09:09.790 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44410.log 2024-03-28T01:09:09.790 INFO:teuthology.orchestra.run.smithi033.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-28T01:09:09.791 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.41834.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz -5 2024-03-28T01:09:09.791 INFO:teuthology.orchestra.run.smithi033.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43025.log 2024-03-28T01:09:09.793 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.44410.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-28T01:09:09.793 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44410.log.gz 2024-03-28T01:09:09.794 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.43025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44074.log 2024-03-28T01:09:09.794 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43025.log.gz 2024-03-28T01:09:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.44074.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44074.log.gz 2024-03-28T01:09:09.799 INFO:teuthology.orchestra.run.smithi194.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-28T01:09:09.806 INFO:teuthology.orchestra.run.smithi194.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-28T01:09:09.813 INFO:teuthology.orchestra.run.smithi194.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-28T01:09:09.824 INFO:teuthology.orchestra.run.smithi033.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-28T01:09:09.868 INFO:teuthology.orchestra.run.smithi194.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-28T01:09:09.872 INFO:teuthology.orchestra.run.smithi194.stderr: 2024-03-28T01:09:09.872 INFO:teuthology.orchestra.run.smithi194.stderr:real 0m0.273s 2024-03-28T01:09:09.872 INFO:teuthology.orchestra.run.smithi194.stderr:user 0m0.815s 2024-03-28T01:09:09.872 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m0.187s 2024-03-28T01:09:09.893 INFO:teuthology.orchestra.run.smithi033.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-28T01:09:09.915 INFO:teuthology.orchestra.run.smithi033.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-28T01:09:09.942 INFO:teuthology.orchestra.run.smithi033.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-28T01:09:09.948 INFO:teuthology.orchestra.run.smithi033.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-28T01:09:09.951 INFO:teuthology.orchestra.run.smithi033.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-28T01:09:09.956 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-03-28T01:09:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m0.356s 2024-03-28T01:09:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m1.103s 2024-03-28T01:09:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.468s 2024-03-28T01:09:09.957 INFO:tasks.ceph:Archiving logs... 2024-03-28T01:09:09.957 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/remote/smithi033/log 2024-03-28T01:09:09.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T01:09:10.248 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/remote/smithi194/log 2024-03-28T01:09:10.249 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T01:09:10.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-03-28T01:09:10.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-03-28T01:09:10.451 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-28T01:09:10.468 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-28T01:09:10.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T01:09:10.476 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T01:09:10.631 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-28T01:09:10.640 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-28T01:09:10.640 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T01:09:10.648 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T01:09:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-03-28T01:09:10.702 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-03-28T01:09:10.707 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T01:09:10.708 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-03-28T01:09:10.708 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 49 +1374us[+1199us] +/- 84ms 2024-03-28T01:09:10.708 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -1029us[-1204us] +/- 49ms 2024-03-28T01:09:10.708 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +229us[ +54us] +/- 63ms 2024-03-28T01:09:10.708 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:09:10.712 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T01:09:10.712 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-03-28T01:09:10.712 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 49 +1415us[+1248us] +/- 84ms 2024-03-28T01:09:10.712 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -1050us[-1218us] +/- 49ms 2024-03-28T01:09:10.712 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +244us[ +244us] +/- 63ms 2024-03-28T01:09:10.712 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T01:09:10.713 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-28T01:09:10.732 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-28T01:09:10.732 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-28T01:09:10.740 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-28T01:09:10.964 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-28T01:09:11.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T01:09:11.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T01:09:11.283 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T01:09:11.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T01:09:11.385 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-28T01:09:11.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-28T01:09:11.404 INFO:teuthology.task.internal:Duration was 778.121365 seconds 2024-03-28T01:09:11.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-28T01:09:11.413 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-28T01:09:11.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T01:09:11.420 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T01:09:11.577 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T01:09:11.593 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T01:09:12.091 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-28T01:09:12.091 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-03-28T01:09:12.092 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T01:09:12.153 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-03-28T01:09:12.153 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T01:09:12.225 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-28T01:09:12.225 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T01:09:12.229 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T01:09:12.309 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-28T01:09:12.309 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T01:09:12.313 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T01:09:12.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-28T01:09:12.498 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-28T01:09:12.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T01:09:12.589 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T01:09:12.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-28T01:09:12.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T01:09:12.704 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T01:09:12.811 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-03-28T01:09:12.813 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-03-28T01:09:12.893 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T01:09:12.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T01:09:12.948 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T01:09:13.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T01:09:13.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-28T01:09:13.013 INFO:teuthology.task.internal:Transferring archived files... 2024-03-28T01:09:13.014 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/remote/smithi033 2024-03-28T01:09:13.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T01:09:13.125 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252/remote/smithi194 2024-03-28T01:09:13.125 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T01:09:13.223 INFO:teuthology.task.internal:Removing archive directory... 2024-03-28T01:09:13.223 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T01:09:13.231 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T01:09:13.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-28T01:09:13.296 INFO:teuthology.task.internal:Not uploading archives. 2024-03-28T01:09:13.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-28T01:09:13.305 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-28T01:09:13.306 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T01:09:13.313 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T01:09:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: 393945 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 28 01:09 /home/ubuntu/cephtest 2024-03-28T01:09:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: 393478 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 28 01:06 /home/ubuntu/cephtest/mnt.0 2024-03-28T01:09:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: 393479 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 28 01:08 /home/ubuntu/cephtest/mnt.0/client.0 2024-03-28T01:09:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: 419135 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 01:08 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-28T01:09:13.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T01:09:13.368 INFO:teuthology.orchestra.run.smithi033.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-28T01:09:13.368 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi033 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-28T01:09:13.377 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-28T01:09:13.386 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-28T01:09:13.395 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi033 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-28T01:09:13.396 INFO:teuthology.nuke:Checking targets against current locks 2024-03-28T01:09:13.403 INFO:teuthology.orchestra.run.smithi194.stdout: 394187 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 01:09 /home/ubuntu/cephtest 2024-03-28T01:09:13.428 DEBUG:teuthology.nuke:shortname: smithi033 2024-03-28T01:09:13.428 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T01:09:13.452 DEBUG:teuthology.nuke:shortname: smithi194 2024-03-28T01:09:13.453 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T01:09:13.456 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 00:47:25.364107', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdDFkbJ8neEy/kRcin/a5EEuG94zZpcsnyExwll9pO+AuwMq9VHSxbQV0SjsbKA7FlR9ml23QSJHNtvur+RG9Q='} 2024-03-28T01:09:13.476 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625252', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 00:47:25.365789', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAWl1fIfCuM29ayfT64fpl1N5+l5xGMs9311mNHxmwUSXAtKLdyj78swsnTh3QWOPEO/KcZnRNnde6WJJnQsBg='} 2024-03-28T01:09:13.514 INFO:teuthology.orchestra.console.smithi033:Power off 2024-03-28T01:09:13.514 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T01:09:13.531 INFO:teuthology.orchestra.console.smithi194:Power off 2024-03-28T01:09:13.532 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T01:09:13.543 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-03-28T01:09:13.543 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T01:09:13.555 DEBUG:teuthology.orchestra.console.smithi194:power off output: Chassis Power Control: Down/Off 2024-03-28T01:09:13.555 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T01:09:13.565 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-03-28T01:09:13.579 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-03-28T01:09:17.566 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T01:09:17.580 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T01:09:17.680 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-03-28T01:09:17.691 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-03-28T01:09:21.682 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T01:09:21.692 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T01:09:21.793 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-03-28T01:09:21.801 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-03-28T01:09:21.895 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-03-28T01:09:21.902 INFO:teuthology.orchestra.console.smithi194:Power off completed 2024-03-28T01:09:22.009 INFO:teuthology.run:Summary data: description: fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} duration: 778.1213648319244 failure_reason: 'Command failed (workunit test client/test.sh) on smithi033 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/client/test.sh''' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3b61430755c448adb3c195d7fdbac6b6 status: fail success: false 2024-03-28T01:09:22.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T01:09:22.100 INFO:teuthology.run:FAIL